127 stories
1 follower

anarchistettin: afunnyfeminist: whatareyoureallyafraidof:   Far...

1 Comment and 16 Shares




  Far too many people have short memories when it comes to the shit their side did.

White supremacists and police have never been asked to commit to nonviolence. That’s only ever been asked of victims.

That’s only ever been asked of victims.

Read the whole story
22 days ago
23 days ago
Share this story
1 public comment
16 days ago
Earlier comment is true, but the latter is bullshit. They get asked to commit to nonviolence all the time. It just doesn't get serious response.

Introducing Watchtower 2.0: The turret becomes a castle

1 Share

Introducing the all new Watchtower – it is absolutely gorgeous, and appears to be rather timely!

Twitter asked their 330 million users to change their password yesterday due to a security snafu, putting privacy and security at the forefront of everyone’s mind once again.

1Password includes Watchtower, with its suite of security tools, making it the easiest and most comprehensive way for you to check the security of all your passwords.

Watchtower report

With a click of a button, Watchtower audits your passwords against a wide range of security vulnerabilities giving you an easy to read report with simple steps on how to fix any issues it finds.

Let’s take a look at some of the defences.

On the lookout for breaches

Watchtower will automatically notify you if there’s been a security breach for a website you use. A bright red bar that’s pretty darn hard to miss will display across the top of the item, prompting you to change the password for that site.

Login showing a breach

Please excuse me while I hop away for a sec and go change that Twitter password. 😀

A vanguard for pwned passwords

Watchtower can check your passwords to see if any have been exposed in a breach. Integrating with Troy Hunt’s haveibeenpwned.com service, your passwords are checked against over 500 million exposed passwords, highlighting any that are found.

Watchtower showing vulnerable passwords

To keep your passwords private, Troy found a brilliant way to check if passwords have been leaked without ever sending your password to his service.

Strong, unique passwords are your greatest defence

Using strong, unique passwords for every website is your surest way to keep safe. When a website is breached and your password compromised, that password can be used to sign in to other websites that use the same one. If you’ve reused that password elsewhere, you’re putting all those sites at risk.

Watchtower not only shows you which of your passwords should be stronger, it also alerts you when you’re using the same passwords for more than one website.

Graph of password strengths

Now would be a great time to use Watchtower to see if you reused your Twitter password for your bank account 😱

A second line of defence

Enabling two-factor authentication (2FA) on websites is a great way to keep your accounts there safe. Watchtower will now let you know about websites you have saved in 1Password that support 2FA, but don’t have it enabled.

Alert showing missing 2FA

This gives you the chance to enable 2FA for those sites. When you enable 2FA, make sure to keep the one-time password in 1Password.

Don’t get caught off guard

Watchtower not only looks out for your passwords, but for you as well. It will now warn you if one of your credit cards, driver’s licenses, or passports are expiring soon, making sure you aren’t scrambling to make last-minute arrangements.

Alert showing expiring passport

Here in Canada you can’t travel internationally if your passport expires within 6 months, so this can be a real life saver if you have that long-planned vacation coming up soon.

Try today with your 1Password membership

Watchtower is available today, so it’s time to give it a try now!

Sign in to your 1Password.com account, select a vault, and click Watchtower in the sidebar to create your report. If you don’t have a 1Password membership, start a free 30-day trial to get started.

Oh, and don’t forget to change your Twitter password :)

The post Introducing Watchtower 2.0: The turret becomes a castle appeared first on AgileBits Blog.

Read the whole story
22 days ago
Share this story

A Basic Guide on AlterationsThe easiest way to improve your...

1 Share

A Basic Guide on Alterations

The easiest way to improve your wardrobe is to take things to the alterations tailor. Since clothes are designed for an idealized body, they often fit everyone and no one in particular at the same time. That means you can often improve the look of your clothes by having things nipped and tucked here and there – taking in the waist or shortening the sleeves, adjusting the length of your trousers so they fall perfectly over your shoes. For a few bucks, you can get off-the-rack clothes that look 90% on their way to being custom made. 

We’ve written dozens of guides on alterations. And while it would be too much to include everything you ought to know in one post, we thought we’d gather the basics into a simplified guide, then include some links to suggested readings from our archive. Pair this with our service directory on how to take care of your clothes and you’ll have most of your clothing services covered. 



Tailors are a bit like barbers. It can take a bit of work to find a good one, and once you do, you should hold on to them for dear life. The quality of your alterations depends on the quality of your tailor – and finding one in your area isn’t always easy. Here are three ways you can go about it:

If you live in a big US city, search clothing boards such as StyleForum, Reddit’s Male Fashion Advice, and Ask Andy About Clothes. Denizens there are often a bit more discerning than most when it comes to quality tailoring, and they’ll be able to point you in the right direction. The downside? Since most members are based in major US cities, you’re more likely to find suggestions if you live in or around a major cosmopolitan center. If you don’t, you may need to try some other strategy. 

Another way is to search for recommendations through local upscale establishments. These can be anything from small, independent menswear boutiques carrying brands you admire to large department stores such as Saks Fifth Avenue and Neiman Marcus. Or they could be single brand shops such as Ralph Lauren or Tom Ford flagships. They may also be high-end hotels, such as the Four Seasons or Ritz Carlton. 

Ask the managers there if they have suggestions. Some clothing stores have in-house tailors they rely on for alterations, but many will send out work to a local shop. Others, such as hotels, may just be plugged into the local network for high-end services. Call a few places, ask for recommendations, and see if one or two names keep popping up. If you can narrow in on a consensus among some trusted sources, it’s likely those places do good work. 

Failing that, there’s always Yelp. Yelp reviewers aren’t always the most reliable, and just because a place is rated well doesn’t necessarily mean it’s good. But trawling Yelp for recommendations will be better than just venturing out on your own. Hedge your bets with a new shop by sending in something inconsequential or easy – say, hemming a pair of cheap chinos or altering an affordable button-up. Once you’ve established they do reliable work, you can slowly move your way up to more complicated jobs. Don’t throw in all your chips at once. Save major surgery jobs for when you’ve confirmed the tailor is good. 

FURTHER READING: Q & Answer, How to Find a Good Tailor and A Three Step Process to Finding Quality Tailors and Dry Cleaners



Let’s start with this: almost any alteration can be done, it just depends on how much you’re willing to pay. At a certain point, the job becomes so complicated and expensive, you’re better off finding something better off-the-rack. This is the difference between converting a car into a drop-top convertible and just buying that convertible outright. Sometimes it’s better to just purchase the thing you want and adjust at the margins. 

Figuring out what can be altered heavily depends on a case-by-case basis. There aren’t any hard and fast rules here, but there are some basic principals you may want to consider.

  • The more complicated the alterations or garment, the more expensive the job. Again, it’s good to keep costs in mind here. Suit jackets and sport coats are more complicated to alter than shirts, and thus alterations are more expensive. Similarly, if a casual jacket has some unusual, hard-to-modify details – such as a leather jacket with unusually placed studs – it may be difficult to work around those parameters. When judging something off the rack, take into account the extremity of the alteration needed, where the alteration needs to take place, and the complexity of the garment’s construction. All of these will factor into your costs.
  • Make sure the garment fits in certain places. You generally want certain areas to fit perfectly off-the-rack. Jackets, shirts, and sweaters, for example, ought to fit perfectly through the chest and shoulders at the outset. If the chest and shoulders don’t fit right from the get-go, put the item back. Similarly, trousers should fit well through the thighs and seat. Getting those areas altered can be difficult, if not impossible. 
  • What are some common alterations? Some alterations are so common, you almost don’t even need to think about them. Suit jackets and sport coats often have a bit of a roll between the shoulder blades, which can be taken out for cheap. Sleeves are commonly taken up (although the job can be a little more difficult with working buttonholes). The waist on shirts and jackets are often nipped; trousers are frequently always hemmed; and the waistband on pants can be taken in or let out within reason. You can also taper trousers and jeans from the knee down, giving them a bit more shape. See this post for a list of common alterations.
  • Consider the material. Wool garments can sometimes be easier to alter, especially if you’re looking to let out things, because the surface nap covers up any holes. Crisp linens, fine cottons, and especially leather, however, will leave visible holes. 
  • Changing details. Often, we’ll get an email from a reader asking if certain details can be altered on a garment. Whether a structured shoulder can be changed into a soft one, or a roped shoulder modified into something more natural looking. Or whether a wide lapel can be changed into something more modest. The answer is often yes, but it’s risky, expensive, and not worth it. Typically, you’ll end up with something that costs a lot of money and still doesn’t look quite right. Stick to simpler jobs. The one exception is taking out the lining in a jacket, at least through the back. 
  • It’s easier to take things in than let things out. The reason is because, in order to let out a garment, you need enough material inside (what tailors call a seam allowance). Most companies don’t build in that much seam allowance, however, because doing so costs money. So, if you’re shopping off-the-rack and something feels a bit tight, your best bet is often to just size up. See this post on our guide for letting out clothes
  • Can This Be Altered? The answer is almost always yes, yes, and yes. But not all tailors have the equipment or skills necessary to do every job, so sometimes you need to find a specialist. See these posts for how to alter leather jackets, sweaters, and neckties (your local alterations shop can likely take care of the rest). 

FURTHER READING: Common Alterations; How Much Should Suit Alterations Cost?; Can Leather Jackets, Knitwear, and Ties Be Altered?Removing the Lining From a Jacket; How to Eliminate Blousing on a Shirt; How Long Should a Suit Jacket or Sport Coat Be?; and Yes, You Can Shorten Shirt Tails



When it comes to getting good alterations, half the battle is finding a good tailor. The other half is having a good eye. While you should always rely on your tailor’s advice, you should also pay attention to some key areas:

Collar Gap: This is numero uno when it comes to making sure your garment fits well – particularly for suits and sport coats, but also casualwear. With few exceptions, such as mountain parkas, all jackets should stay glued onto your neck, even when you’re moving around (again, within reason). A collar gap is when the jacket hovers from your neck, suggesting that maybe the cut and fit aren’t quite right (see above). Jesse wrote an excellent explainer a few years ago

Unfortunately, it’s not clear whether a collar gap can be fixed. “A lot depends on the exact cause, the severity, and the make of the jacket,” says Chris Despos, a bespoke tailor in Chicago. “If the shoulders need to be squared up, there’s only so much you can do before you cause other kinds of issues. If the back needs to be shifted, you’re limited by how much extra cloth is available at the hem. It’s hard to diagnosis these things without seeing a client in person.“ Your best bet, says Chris, is to take things into a local alterations tailor and be prepared to return the garment if things don’t work out. 


Shoulder Divot: The dreaded shoulder divot was once the mark of pure shame on clothing boards. And it’s still one of the most common fit defects on suits and sport coats. The term refers to the small indentation that can happen on the upper part of the sleeveheads, which ruins the otherwise clean line running down from the jacket’s shoulder and into the sleeves (see the photo above).

People often think this happens because the shoulders are too wide, but it’s actually the opposite. While shoulder divots can occur from poor workmanship or design, they’re often because the jacket’s shoulders are too narrow for the wearer. You can get this fixed at a local alterations tailor, but the job is often complicated, expensive, and can cause other issues (letting the jacket out along the back seam, for example, can cause mismatched patterns). Instead, just size up. Tutto Fatto a Mano has a great post about this.

Sleeve Pitch: For suits and sport coats, sometimes the sleeves don’t hang smoothly because their rotation – or pitch – don’t match the natural pitch of your arms. So, when you’re standing naturally, if your arm is a little too pushed back or forward, it can cause wrinkling along the front or back of the sleeve. A StyleForum member once put together a nice little illustration showing this effect. The good news is that a tailor can usually alter this for you. 


The Back of Trousers: When you’re at your tailor’s, utilize that three-way mirror and see how your trousers hang from the back – it’s one of the easiest things to miss. You can always see how trousers hang from the front, but it’s often the seat and the backs of the legs that have issues. These areas should drape cleanly, like you see here on Panta’s custom-made trousers. To be sure, you probably can’t get something to fit that exact off-the-rack, but it’s better to be closer to the ideal than not. The good news is that these issues can sometimes be adjusted by a local alterations tailor, but a lot depends on the exact cause of the problems, the severity of the issues, and the make of the garment (much like a coat’s collar gap). See our post on common fit issues with trousers. 

Cuffs and Breaks: If you’re sending in pants, decide beforehand how you feel about cuffs and breaks. Whether you cuff your trousers is a personal choice, although they should be left off the most formal of suits, such as tuxedos. We have a full guide on cuffs here. Breaks, on the other hand, are a bit more by-the-rules. The break of your trousers is where the hem touches the shoes, and unless you’re wearing something fashion forward or avant-garde, you should avoid things that are either pooling around your ankles or cropped. Instead, go for either a full break, slight break, or no break at all – but make sure the hem of your pants are still touching the shoes. Again, we have a full guide on breaks here

Darting Shirts: One of the most common alterations jobs is slimming down a shirt. And depending on your body, you may find that you can’t get as much out as you want through the side seams alone. In such cases, you can consider darting the back. You can see an example of a darted shirt above (the faint lines near the sides of the shirt are darts). 

Darts are folds that have been pinched and then sewn into a garment. They’re basically a way to add shape – turning a flat piece of cloth into something with curves. When put into the back of a shirt, they do two things. First, they’ll take out the fullness at the lower back, helping reveal that hollowed shape. As a result, you’ll have a bit more of a sculpted look. Second, they’ll help slim down the shirt when the tailor can’t take any more out of the side seams. See here for our full guide on darting shirts. (Pro tip: Tutto Fatto a Mano has a cool post here about darting jeans so they better cover a prominent seat. Maybe something your local tailor can also do for you). 

FURTHER READING: Can a Collar Gap be Fixed?The Details of Sleeve Pitch; Deciding Whether Your Trousers Should be CuffedHow Much Should My Trousers BreakThe Difference Between Darts and Side Seams on Shirts; and How to Eliminate Blousing on a Shirt



Rely On Your Tailor’s Advice. Don’t micromanage the process too much. If you get a good tailor, he or she should be able to guide you towards better decisions. Rely on them for their advice. They’re the professional, after all. 

Pay Attention to Fit. With that said, the tailor isn’t here to style you. Go into this process with an eye for how you’d like clothes to fit (we have tons of guides). You should also decide on things such as cuffing and breaks, as mentioned above. And be wary of going too slim. A tailor can always take in a garment, at least as much as your body will allow, but that doesn’t mean it’ll look good. Getting clothes slimmed down too much is the most common mistake of new and overeager customers. 

Take Things Slowly. Can’t decide between cuffing and not cuffing trousers? When in doubt, always cuff. Because while you can always remove them, you can’t put cuffs into trousers if there’s not enough material. Similarly, if you’re unsure about a certain alteration, err on the side of caution. Certain things can’t be reversed, so try living with a detail or cut for a while before deciding how you feel about it. 

Wear the Right Clothes. When bringing things to your tailor, you’ll typically try on the garment in front of them, so he or she can pin and chalk things at the right places. That means you should be wearing the kind of clothes you plan to wear with the item. So, if you’re bringing in a suit, arrive in your dress shoes and dress shirt. If you’re sending in a casual coat, bring along a sweater. This way, you and your tailor can get a better sense of what needs to be done in order to get these outfits to look right. 

Account for Movement. Don’t forget to account for movement. Shirt sleeves should be long enough so that the cuff stays at your wrist when you move your arms. Linen garments wrinkle, which means it’s ok for sleeves and trousers to be a little longer than usual – they’ll come up to the right length once you wear them for a few hours. And trousers ride up a bit when you walk, so be careful of getting things too short. Otherwise, too much of your ankle will show when you hit your stride or sit down. 

The Makeshift Shoe Horn. Sometimes, when changing in and out of pants, your tailor may not have a shoehorn. In these cases, use something like your credit card. A thick plastic card, when placed between your heel and shoe, basically does the same trick.



Put This On’s team is spread across five cities, so we thought we’d put together a list of the tailors we use. To be sure, these aren’t the only reputable establishments in these areas, just the tailors we’ve personally relied on for years and can vouch for. If you happen to live in or near these cities, and don’t already have a tailor, consider these places. We think they do exceptional work. 

Read the whole story
53 days ago
Share this story

Here's Where That Ram Ad Really Got Martin Luther King Jr. Wrong

1 Share

Ram debuted an ad at the Super Bowl last night featuring the appropriated words of the Reverend Dr. Martin Luther King Jr. If you haven’t seen it, it was bad and dumb, and has been derided pretty much across the board. But it’s not just the use of King’s words to sell trucks that got it wrong—it’s what the rest of…


Read the whole story
110 days ago
Share this story

Dynamic Inventory: Past, Present & Future

1 Share

In Red Hat Ansible Engine 2.4, we made some changes to how inventory works. We introduced a new cli tool, and added an inventory plugin type.

The goal of this plugin type, was well, to make Ansible Engine even more pluggable. All kidding aside, we wanted to provide Ansible Engine content authors and Ansible Engine users a new way to visualize their target inventory.  Using the ansible-inventory command,  the targeted inventory will be listed with the details of the hosts in that inventory, and the hosts groups.

For example:

[thaumos@ecb51a545078 /]# ansible-inventory -i ~/Development/playbooks/inventory/prod.aws_ec2.yml --list
    "_meta": {
        "hostvars": {
            "ec2-5x-xx-x-xxx.us-west-2.compute.amazonaws.com": {
                "AmiLaunchIndex": 2,
                "Architecture": "x86_64",
                "BlockDeviceMappings": [
                        "DeviceName": "/dev/sda1",
                        "Ebs": {
                            "AttachTime": "2017-12-13T15:40:19+00:00",
                            "DeleteOnTermination": false,
                            "Status": "attached",
                            "VolumeId": "vol-0514xxx"
                "ClientToken": "",
                "EbsOptimized": false,
                "Hypervisor": "xen",
                "ImageId": "ami-0c2aba6c",
                "InstanceId": "i-009xxxx3",
                "InstanceType": "t2.micro",
                "KeyName": "blogKey",
                "LaunchTime": "2017-12-13T15:40:18+00:00",
                "Monitoring": {
                    "State": "disabled"
                "NetworkInterfaces": [
                        "Association": {
                            "IpOwnerId": "amazon",
                            "PublicDnsName": "ec2-5x-xx-x-xxx.us-west-2.compute.amazonaws.com",
                            "PublicIp": "5x.xx.x.xxx"
                        "Attachment": {
                            "AttachTime": "2017-12-13T15:40:18+00:00",
                            "AttachmentId": "eni-attach-97c4xxxx",
                            "DeleteOnTermination": true,
                            "DeviceIndex": 0,
                            "Status": "attached"
                        "Description": "",
                        "Groups": [
                                "GroupId": "sg-e63xxxxd",
                                "GroupName": "blogGroup"
                        "Ipv6Addresses": [],
                        "MacAddress": "02:50:99:0b:e8:e8",
                        "NetworkInterfaceId": "eni-eaxxxxxx",
                        "OwnerId": "xxxxxxxx",
                        "PrivateDnsName": "ip-1xx-xx-xx-x.us-west-2.compute.internal",
                        "PrivateIpAddress": "1xx.xx.xx.x",
                        "PrivateIpAddresses": [
                                "Association": {
                                    "IpOwnerId": "amazon",
                                    "PublicDnsName": "ec2-5x-xx-x-xxx.us-west-2.compute.amazonaws.com",
                                    "PublicIp": "52.xx.x.xxx"
                                "Primary": true,
                                "PrivateDnsName": "ip-1xx-xx-xx-x.us-west-2.compute.internal",
                                "PrivateIpAddress": "1xx.xx.xx.x"
                        "SourceDestCheck": true,
                        "Status": "in-use",
                        "SubnetId": "subnet-1xxx279",
                        "VpcId": "vpc-4cxxxx29"
                "Placement": {
                    "AvailabilityZone": "us-west-2a",
                    "GroupName": "",
                    "Tenancy": "default"
                "PrivateDnsName": "ip-1xx-xx-xx-x.us-west-2.compute.internal",
                "PrivateIpAddress": "1xx.xx.x.x",
                "ProductCodes": [],
                "PublicDnsName": "ec2-5x-xx-x-xxx.us-west-2.compute.amazonaws.com",
                "PublicIpAddress": "5x.xx.x.xxx",
                "RootDeviceName": "/dev/sda1",
                "RootDeviceType": "ebs",
                "SecurityGroups": [
                        "GroupId": "sg-e6xxxxxx",
                        "GroupName": "blogGroup"
                "SourceDestCheck": true,
                "State": {
                    "Code": 16,
                    "Name": "running"
                "StateTransitionReason": "",
                "SubnetId": "subnet-1cxxxxxx",
                "Tags": [
                        "Key": "env",
                        "Value": "Production"
                        "Key": "Name",
                        "Value": "blogKey"
                "VirtualizationType": "hvm",
                "VpcId": "vpc-4cexxxxx"
    "all": {
        "children": [
    "aws_ec2_instance_type_t2_micro": {
        "hosts": [
    "aws_ec2_tag_value_blogger": {
        "hosts": [
    "ungrouped": {}
Another nifty thing the tool does is show a tree or graph of what the inventory looks like:
[thaumos@ecb51a545078 /]# ansible-inventory -i ~/Development/playbooks/inventory/prod.aws_ec2.yml --graph
  |  |--ec2-5x-xx-x-xxx.us-west-2.compute.amazonaws.com
  |  |--ec2-5x-xx-x-xxx.us-west-2.compute.amazonaws.com

All of this data is being provided by our new AWS EC2 inventory plugin!  Inventory plugins are a much simpler interface for Ansible Engine content providers to interact with the underlying inventory components. This inventory plugin will be provided as a tech-preview. Please provide feedback on it!

Most inventory plugins utilize simple yaml based configuration files to activate the plugin[1] and fine tune the data collected.  Here's an example:

    plugin: aws_ec2
    boto_profile: default
    regions: # populate inventory with instances in these regions
      - us-west-2
    # makes a group for instances that have the tag values 'thaumos' and 't2.micro' instance type.
      - tag-value=thaumos
      - instance-type=t2.micro
      tag:Name: blogger
      tag:env: Production
      instance-state-name: running
    strict_permissions: False

These plugins also have access to the inventory cache that is being built in Ansible Engine 2.5, also in tech preview.  This inventory cache allows for users to combine the powers, so to speak, of other inventory plugins.  For instance, imagine using any of the cloud inventory plugins to pull data about the instances and the constructed inventory plugin and jinja to build groups as decided by the user!  This approach is much more flexible than before; where a content developer would have to build grouping into the dynamic inventory script itself.

I hope the information in this blog post is a great teaser to up and coming features.  I plan on doing little teasers like these more often as we have new stuff coming in Ansible Engine.  These posts will also coincide with a webinar.  The webinar for this blog post can be found here!

Continue Automating all the things!

[1]: Default behaviour in configuration.  Remove 'auto' from INVENTORY_ENABLED in ansible config to stop auto-loading of inventory plugins.


Read the whole story
117 days ago
Share this story

Terraforming 1Password

1 Share

A few days ago I posted this tweet:

The tweet generated quite a bit of interest from people running or managing their own services and I thought I would share some of the cool things we are working on.

This post will go into technical details and I apologize in advance if I explain things too quickly. I tried to make up for this by including some pretty pictures but most of them ended up being code snippets. 🙂

1Password and AWS

1Password is hosted by Amazon Web Services (AWS). We’ve been using AWS for several years now and it is amazing how easy it was to scale our service from zero users three years ago to several million happy customers today.

AWS has many geographical regions. Each region consists of multiple independent data centres located closely together. We are currently using three regions:

  • N. Virginia, USA us-east-1
  • Montreal, Canada ca-central-1
  • Frankfurt, Germany eu-central-1

In each region we have four environments running 1Password:

  • production
  • staging
  • testing
  • development

If you are counting, that’s 12 environments across three regions, including three production environments: 1password.com, 1password.ca, and 1password.eu.

Every 1Password environment is more or less identical and includes these components:

  • Virtual Private Cloud
  • Amazon Aurora database cluster
  • Caching (Redis) clusters
  • Subnets
  • Routing tables
  • Security roles
  • IAM permissions
  • Auto scaling groups
  • Elastic Compute Cloud (EC2) instances
  • Elastic Load Balancers (ELB)
  • Route53 DNS (both internal and external)
  • Amazon S3 buckets
  • CloudFront distributions
  • Key Management System (KMS)

Here is a simplified diagram:


As you can see, there are many components that work together to provide 1Password service. One of the reasons it is so complex is the need for high availability. Most of the components are deployed as a cluster to make sure there are at least two of each: database, cache, server instance, and so on.

Furthermore, every AWS region has at least two data centres that are also known as Availability Zones (AZs) – you can see them in blue in the diagram above. Every AZ has its own independent power and network connections. For example, Canadian region ca-central-1 has two data centres: ca-central-1a and ca-central-2a.

If we deployed all 1Password components into just a single Availability Zone, then we would not be able to achieve high availability because a single problem in the data centre would take 1Password offline. This is why when 1Password services are deployed in a region, we make sure that every component has at least one backup in the neighbouring data centre. This helps to keep 1Password running even when there’s a problem in one of the data centres.

Infrastructure as Code

It would be very difficult and error-prone to manually deploy and maintain 12 environments, especially when you consider that each environment consists of at least 50 individual components.

This is why so many companies today switched from updating their infrastructure manually and embraced Infrastructure as Code. With Infrastructure as Code, the hardware becomes software and can take advantage of all software development best practices. When we apply these practices to infrastructure, every server, every database, every open network port can be written in code, committed to GitHub, peer-reviewed, and then deployed and updated as many times as necessary.

For AWS customers, there are two major languages that could be used to describe and maintain the infrastructure:

CloudFormation is a great option for many AWS customers and we successfully used it to deploy 1Password environments for over two years. At the same time we wanted to move to Terraform as our main infrastructure tool for several reasons:

  • Terraform has a simpler and more powerful language (HCL) that makes it easier to write and review code.
  • Terraform has the concept of resource providers that allows us to manage resources outside of Amazon Web Services, including services like DataDog and PagerDuty, which we rely on internally.
  • Terraform is completely open source and that makes it easier to understand and troubleshoot.
  • We are already using Terraform for smaller web apps at AgileBits and it makes sense to standardize on a single tool.

Compared to the JSON or YAML files used by CloudFormation, Terraform HCL is both a more powerful and a more readable language. Here is a small example with a snippet that defines a subnet for the application servers. As you can see, the Terraform code a quarter of the size, more readable, and easier to understand.


"B5AppSubnet1": {
    "Type": "AWS::EC2::Subnet",
    "Properties": {
        "CidrBlock": { "Fn::Select" : ["0", { "Fn::FindInMap" : [ "SubnetCidr", { "Ref" : "Env" }, "b5app"] }] },
        "AvailabilityZone": { "Fn::Select" : [ "0", { "Fn::GetAZs" : "" } ]},
        "VpcId": { "Ref": "Vpc" },
        "Tags": [
            { "Key" : "Application", "Value" : "B5" },
            { "Key" : "env", "Value": { "Ref" : "Env" } },
            { "Key" : "Name", "Value": { "Fn::Join" : ["-", [ {"Ref" : "Env"}, "b5", "b5app-subnet1"]] } }

"B5AppSubnet2": {
    "Type": "AWS::EC2::Subnet",
    "Properties": {
        "CidrBlock": { "Fn::Select" : ["1", { "Fn::FindInMap" : [ "SubnetCidr", { "Ref" : "Env" }, "b5app"] }] },
        "AvailabilityZone": { "Fn::Select" : [ "1", { "Fn::GetAZs" : "" } ]},
        "VpcId": { "Ref": "Vpc" },
        "Tags": [
            { "Key" : "Application", "Value" : "B5" },
            { "Key" : "env", "Value": { "Ref" : "Env" } },
            { "Key" : "Name", "Value": { "Fn::Join" : ["-", [ {"Ref" : "Env"}, "b5", "b5app-subnet2"]] } }

"B5AppSubnet3": {
    "Type": "AWS::EC2::Subnet",
    "Properties": {
        "CidrBlock": { "Fn::Select" : ["2", { "Fn::FindInMap" : [ "SubnetCidr", { "Ref" : "Env" }, "b5app"] }] },
        "AvailabilityZone": { "Fn::Select" : [ "2", { "Fn::GetAZs" : "" } ]},
        "VpcId": { "Ref": "Vpc" },
        "Tags": [
            { "Key" : "Application", "Value" : "B5" },
            { "Key" : "env", "Value": { "Ref" : "Env" } },
            { "Key" : "Name", "Value": { "Fn::Join" : ["-", [ {"Ref" : "Env"}, "b5", "b5app-subnet3"]] } }


resource "aws_subnet" "b5app" {
  count             = "${length(var.subnet_cidr["b5app"])}"
  vpc_id            = "${aws_vpc.b5.id}"
  cidr_block        = "${element(var.subnet_cidr["b5app"],count.index)}"
  availability_zone = "${var.az[count.index]}"

  tags {
    Application = "B5"
    env         = "${var.env}"
    type        = "${var.type}"
    Name        = "${var.env}-b5-b5app-subnet-${count.index}"

Terraform has another gem of a feature that we rely on: terraform plan. It allows us to visualize the changes that will happen to the environment without performing them.

For example, here is what would happen if we change the server instance size from t2.medium to t2.large.

Terraform Plan Output

# Terraform code changes
# variable "instance_type" {
#    type        = "string"
# -  default     = "t2.medium"
# +  default     = "t2.large"
#  }

$ terraform plan 
Refreshing Terraform state in-memory prior to plan...


An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

-/+ module.b5site.aws_autoscaling_group.asg (new resource required)
      id:                                 "B5Site-prd-lc20180123194347404900000001-asg" =>  (forces new resource)
      arn:                                "arn:aws:autoscaling:us-east-1:921352000000:autoScalingGroup:32b38032-56c6-40bf-8c57-409e9e4a264a:autoScalingGroupName/B5Site-prd-lc20180123194347404900000001-asg" => 
      default_cooldown:                   "300" => 
      desired_capacity:                   "2" => "2"
      force_delete:                       "false" => "false"
      health_check_grace_period:          "300" => "300"
      health_check_type:                  "ELB" => "ELB"
      launch_configuration:               "B5Site-prd-lc20180123194347404900000001" => "${aws_launch_configuration.lc.name}"
      load_balancers.#:                   "0" => 
      max_size:                           "3" => "3"
      metrics_granularity:                "1Minute" => "1Minute"
      min_size:                           "2" => "2"
      name:                               "B5Site-prd-lc20180123194347404900000001-asg" => "${aws_launch_configuration.lc.name}-asg" (forces new resource)
      protect_from_scale_in:              "false" => "false"
      tag.#:                              "4" => "4"
      tag.1402295282.key:                 "Application" => "Application"
      tag.1402295282.propagate_at_launch: "true" => "true"
      tag.1402295282.value:               "B5Site" => "B5Site"
      tag.1776938011.key:                 "env" => "env"
      tag.1776938011.propagate_at_launch: "true" => "true"
      tag.1776938011.value:               "prd" => "prd"
      tag.3218409424.key:                 "type" => "type"
      tag.3218409424.propagate_at_launch: "true" => "true"
      tag.3218409424.value:               "production" => "production"
      tag.4034324257.key:                 "Name" => "Name"
      tag.4034324257.propagate_at_launch: "true" => "true"
      tag.4034324257.value:               "prd-B5Site" => "prd-B5Site"
      target_group_arns.#:                "2" => "2"
      target_group_arns.2352758522:       "arn:aws:elasticloadbalancing:us-east-1:921352000000:targetgroup/prd-B5Site-8080-tg/33ceeac3a6f8b53e" => "arn:aws:elasticloadbalancing:us-east-1:921352000000:targetgroup/prd-B5Site-8080-tg/33ceeac3a6f8b53e"
      target_group_arns.3576894107:       "arn:aws:elasticloadbalancing:us-east-1:921352000000:targetgroup/prd-B5Site-80-tg/457e9651ad8f1af4" => "arn:aws:elasticloadbalancing:us-east-1:921352000000:targetgroup/prd-B5Site-80-tg/457e9651ad8f1af4"
      vpc_zone_identifier.#:              "2" => "2"
      vpc_zone_identifier.2325591805:     "subnet-d87c3dbc" => "subnet-d87c3dbc"
      vpc_zone_identifier.3439339683:     "subnet-bfe16590" => "subnet-bfe16590"
      wait_for_capacity_timeout:          "10m" => "10m"

-/+ module.b5site.aws_launch_configuration.lc (new resource required)
      id:                                 "B5Site-prd-lc20180123194347404900000001" =>  (forces new resource)
      associate_public_ip_address:        "false" => "false"
      ebs_block_device.#:                 "0" => 
      ebs_optimized:                      "false" => 
      enable_monitoring:                  "true" => "true"
      iam_instance_profile:               "prd-B5Site-instance-profile" => "prd-B5Site-instance-profile"
      image_id:                           "ami-263d0b5c" => "ami-263d0b5c"
      instance_type:                      "t2.medium" => "t2.large" (forces new resource)
      key_name:                           "" => 
      name:                               "B5Site-prd-lc20180123194347404900000001" => 
      name_prefix:                        "B5Site-prd-lc" => "B5Site-prd-lc"
      root_block_device.#:                "0" => 
      security_groups.#:                  "1" => "1"
      security_groups.4230886263:         "sg-aca045d8" => "sg-aca045d8"
      user_data:                          "ff8281e17b9f63774c952f0cde4e77bdba35426d" => "ff8281e17b9f63774c952f0cde4e77bdba35426d"

Plan: 2 to add, 0 to change, 2 to destroy.

Overall, Terraform is a pleasure to work with and that makes a huge difference in our daily lives. DevOps people like to enjoy their lives too. 🙌

Migration from CloudFront to Terraform

It is possible to simply import the existing AWS infrastructure directly into Terraform but there are certain downsides to it. We found that naming conventions are quite different and that would make it more challenging to maintain our environments in the future. Also, a simple import would not allow us to use the new Terraform features. For example, instead of hard-coding the identifiers of Amazon Machine Images used for deployment we started using aws_ami to dynamically find the most recent image:


data "aws_ami" "bastion_ami" {
  most_recent = true
  filter {
    name   = "architecture"
    values = ["x86_64"]
  filter {
    name   = "name"
    values = ["bastion-*"]
  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  name_regex = "bastion-.*"
  owners     = [92135000000]

It took us a couple weeks to write the code from scratch. After we had the same infrastructure described in Terraform, we recreated all non-production environments where downtime wasn’t an issue. This also allowed us to create a complete checklist of all the steps required to migrate the production environment.

Finally, on January 21, 2018, we completely recreated 1Password.com. We had to bring the service offline during the migration. Most of our customers were not affected by the downtime because the 1Password apps are designed to function even when the servers are down or when an Internet connection is not available. Unfortunately, our customers who needed to access the web interface during that time were unable to do so, and we apologize for the interruption. Most of the 2 hours and 39 minutes of downtime was related to data migration. The 1Password.com database is just under 1TB in size (not including documents and attachments) and it took almost two hours to complete the snapshot and restore operations.

We are excited to finally have all our development, test, staging, and production environments managed with Terraform. There are many new features and improvements we have planned for 1Password and it will be fun to review new infrastructure pull requests on GitHub!

I remember when we were starting out we hosted our very first server with 1&1. It would have taken weeks to rebuild the very simple environment there. The world has come a long way since we first launched 1Passwd 13 years ago. I am looking forward to what the next 13 years will bring! 😃


A few questions and suggestions about the migration came up on Twitter:

By “recreating” you mean building out a whole new VPC with Terraform? Couldn’t you build it then switch existing DNS over for much less down time?1

This is pretty much what we ended up doing. Most of the work was performed before the downtime. Then we updated the DNS records to point to the new VPC.

Couldn’t you’ve imported all online resources? Just wondering.2

That is certainly possible and it would have allowed us to avoid downtime. Unfortunately, it also requires manual mapping of all existing resources. Because of that, it’s hard to test and the chance of a human error is high – and we know humans are pretty bad at this. As a wise person on Twitter said: “If you can’t rebuild it, you can’t rebuild it“.

If you have any questions, let us know in the comments, or ask me (@roustem) and Tim (@stumyp), our Beardless Keeper of Keys and Grounds, on Twitter.

Read the whole story
121 days ago
Share this story
Next Page of Stories