Provision a Bitbucket SmartMirror Instance using AWS CloudFormation
This is a follow-up to my previous article titled “Bitbucket SmartMirror Automation” which details how to provision a Bitbucket SmartMirror using the AWS UserData code block in EC2 Launch Instance. This article goes a step farther to more fully automate the process using AWS CloudFormation.
Assumptions:
This post assumes that you have a fairly advanced knowledge of AWS including CloudFormation, EC2, Load Balancers, AWS Certificate Manager, Security Groups, etc.. Please reference the AWS documentation for additional info on those topics.
Prerequisites:
- Create an AWS Classic internal Load Balancer with listeners: (22 -> 7999, 7999 -> 7999, 443 -> 7990) # Attach an AWS cert for SSL termination on 443. (‘Classic’ is the only LB which allows a mix of both HTTPS & TCP ports.)
- Your LB Security Group should allow access to ports 22, 7999 and 443 from your internal network CIDR’s or specific IP’s.
- Register an internal DNS CNAME which points to your LB’s DNS address. The CloudFormation script will prompt for this when creating the stack.
- In addition to the FQDN of the SmartMirror, the script will also require entry of a friendly name to be used for your SmartMirror…ie US-East-mirror1 or Mumbai-mirror1.
- You’ll need to provide the name of the SSH key that you would like to associate with the EC2 instance.
- You’ll need to know the name of the target VPC as well as the target subnet for the EC2 instance.
- You’ll be prompted to enter the volume size for the EC2 instance. Make sure this is large enough to store all of the repositories that you intend to mirror. Defaults to 100Gb.
---
AWSTemplateFormatVersion: 2010-09-09
Description: "Launch a Bitbucket SmartMirror EC2 instance."
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: EC2 Instance Setup
Parameters:
- Ec2NameTag
- KeyName
- Ec2VolumeSize
- WhichVPC
- WhichSubnet
- Label:
default: Bitbucket SmartMirror Setup
Parameters:
- BitbucketVersion
- FqdnOfSmartMirror
- FriendlyNameOfSmartMirror
- UpstreamBitbucketUrl
Parameters:
Ec2NameTag:
Description: "The name to be given to this EC2 instance....ie Bitbucket-SmartMirror"
Type: String
KeyName:
Description: "Name of an existing EC2 KeyPair to enable SSH access into the server."
Type: AWS::EC2::KeyPair::KeyName
Ec2VolumeSize:
Default: 100
Description: "Size of EC2 root volume in Gb. Make it large enough to hold all of the repos you intend to mirror."
Type: Number
WhichVPC:
Default: ""
Description: "Choose the VPC in which this EC2 instance should reside."
Type: AWS::EC2::VPC::Id
WhichSubnet:
Description: "Choose the Subnet in which this EC2 instance should reside."
Type: AWS::EC2::Subnet::Id
BitbucketVersion:
Default: "7.15.1"
AllowedPattern: '([^1234]\.\d+\.\d+(-?.*))'
ConstraintDescription: "Must be a valid Bitbucket version number. For example: 7.15.0"
Description: "Version of Bitbucket SmartMirror to install. Find valid versions at http://go.atlassian.com/bbs-releases"
Type: String
FqdnOfSmartMirror:
Description: "The FQDN of the DNS CNAME that you've assigned for this SmartMirror."
Type: String
FriendlyNameOfSmartMirror:
Description: "The friendly name to be used for this SmartMirror....ie US-East-mirror1 or Mumbai-mirror1"
Type: String
UpstreamBitbucketUrl:
Description: "The FQDN of the upstream Bitbucket server to which this SmartMirror will connect....ie bitbucket.example.com"
Type: String
Mappings:
RegionMap:
us-east-1:
AMI: ami-087c17d1fe0178315
eu-west-1:
AMI: ami-0d1bf5b68307103c2
ap-south-1:
AMI: ami-0a23ccb2cdd9286bb
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
config:
packages:
yum:
git: []
Properties:
InstanceType: t3.large
ImageId:
Fn::FindInMap:
- RegionMap
- !Ref AWS::Region
- AMI
SecurityGroupIds:
- !Ref SmartMirrorSecurityGroup
SubnetId: !Ref WhichSubnet
Tags:
- Key: Name
Value: !Ref Ec2NameTag
KeyName: !Ref KeyName
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
VolumeType: gp2
VolumeSize: !Ref Ec2VolumeSize
DeleteOnTermination: "false"
UserData:
'Fn::Base64':
!Sub |
#!/bin/bash -xe
FQDN_OF_SMARTMIRROR=${FqdnOfSmartMirror}
FRIENDLY_NAME_OF_SMARTMIRROR=${FriendlyNameOfSmartMirror}
UPSTREAM_BB_URL=${UpstreamBitbucketUrl}
BITBUCKETVERSION=${BitbucketVersion}
# Ensure AWS CFN Bootstrap is the latest
yum install -y aws-cfn-bootstrap
# Install the files and packages from the metadata
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}
# Install Bitbucket configured as a SmartMirror
mkdir -p /var/atlassian/application-data/bitbucket /opt/atlassian
cd /opt/atlassian
wget https://www.atlassian.com/software/stash/downloads/binary/atlassian-bitbucket-$BITBUCKETVERSION-x64.bin
chmod 750 atlassian-bitbucket-$BITBUCKETVERSION-x64.bincat <<EOF > response.varfile
app.bitbucketHome=/var/atlassian/application-data/bitbucket
app.defaultInstallDir=/opt/atlassian/bitbucket/$BITBUCKETVERSION
app.install.service$Boolean=true
httpPort=7990
installation.type=MIRROR_INSTALL
sys.adminRights$Boolean=true
sys.languageId=en
launch.application$Boolean=false
EOF./atlassian-bitbucket-$BITBUCKETVERSION-x64.bin -q -varfile response.varfilecat <<EOF >> /var/atlassian/application-data/bitbucket/shared/bitbucket.properties
server.proxy-name=$FQDN_OF_SMARTMIRROR
setup.baseUrl=https://$FQDN_OF_SMARTMIRROR
setup.displayName=$FRIENDLY_NAME_OF_SMARTMIRROR
plugin.mirroring.upstream.url=https://$UPSTREAM_BB_URL
plugin.mirroring.upstream.type=server
server.secure=true
server.require-ssl=true
server.scheme=https
server.proxy-port=443
EOFsed -i.orig 's/# umask 0027/umask 0027/; s/JVM_MINIMUM_MEMORY=512m/JVM_MINIMUM_MEMORY=2g/; s/JVM_MAXIMUM_MEMORY=1g/JVM_MAXIMUM_MEMORY=2g/' /opt/atlassian/bitbucket/$BITBUCKETVERSION/bin/_start-webapp.shservice atlbitbucket start
SmartMirrorSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Open Ports 22, 7990 and 7999
GroupName: SmartMirror-sg
VpcId: !Ref WhichVPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '7990'
ToPort: '7990'
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '7999'
ToPort: '7999'
CidrIp: 0.0.0.0/0
Outputs:
Website:
Description: The Public DNS for the EC2 Instance
Value: !Sub 'http://${EC2Instance.PublicDnsName}'
Create your AWS CloudFormation Stack and when prompted, use the template script above. You’ll be prompted to enter the data listed in the prerequisites above, then the stack will create your Bitbucket SmartMirror.
IMPORTANT: After the stack completes, edit the security group (SmartMirror-sg) that was created for your instance, replacing 0.0.0.0/0 with specific IP’s or CIDR ranges instead of allowing access to all.
Now add your new instance to your Load Balancer (instances tab).
Login to your Bitbucket Data Center cluster as a user with Admin privileges and click on the “Administration” cog:
…then click on the “Mirrors” link on the left:
If all went well, you should see a Mirror Request from your new EC2 instance waiting to be accepted. Once accepted, you can choose which projects to mirror, or mirror ALL projects (hopefully you’ve allocated enough storage on your mirror for this).
I hope this post makes creating additional Bitbucket SmartMirrors a little quicker and simpler as it did for me. After configuring my DNS and AWS Load Balancer, I’m able to provision a new mirror in just a couple of minutes.
