Note: I originally wrote and published this article as part of the Automating Your PHP Application Deployment Process with Ansible tutorial series for the Digital Ocean Community.
Introduction
This tutorial is the second in a series about deploying PHP applications using Ansible on Ubuntu 14.04. The first tutorial covers the basic steps for deploying an application, and is a starting point for the steps outlined in this tutorial.
In this tutorial we will cover setting up SSH keys to support code deployment/publishing tools, configuring the system firewall, provisioning and configuring the database (including the password!), and setting up task schedulers (crons) and queue daemons. The goal at the end of this tutorial is for you to have a fully working PHP application server with the aforementioned advanced configuration.
Like the last tutorial, we will be using the Laravel framework as our example PHP application. However, these instructions can be easily modified to support other frameworks and applications if you already have your own.
Prerequisites
This tutorial follows on directly from the end of the first tutorial in the series, and all of the configuration and files generated for that tutorial are required. If you haven’t completed that tutorial yet, please do so first before continuing with this tutorial.
Step 1 — Switching the Application Repository
In this step, we will update the Git repository to a slightly customized example repository.
Because the default Laravel installation doesn’t require the advanced features that we will be setting up in this tutorial, we will be switching the existing repository from the standard repository to an example repository with some debugging code added, just to show when things are working. The repository we will use is located at https://github.com/do-community/do-ansible-adv-php.
If you haven’t done so already, change directories into ansible-php from the previous tutorial.
cd ~/ansible-php/Open up our existing playbook for editing.
nano php.ymlFind and update the “Clone git repository” task, so it looks like this.
- name: Clone git repository
git: >
dest=/var/www/laravel
repo=https://github.com/do-community/do-ansible-adv-php
update=yes
version=example
sudo: yes
sudo_user: www-data
register: clonedSave and run the playbook.
ansible-playbook php.yml --ask-sudo-passWhen it has finished running, visit your Droplet in your web browser (i.e. http://your_server_ip/). You should see a message that says “could not find driver”.
This means we have successfully swapped out the default repository for our example repository, but the application cannot connect to the database. This is what we expect to see here, and we will install and set up the database later in the tutorial.
Step 2 — Setting up SSH Keys for Deployment
In this step, we will set up SSH keys that can be used for application code deployment scripts.
While Ansible is great for maintaining configuration and setting up servers and applications, tools like Envoy and Rocketeer are often used to push code changes onto your server and run application commands remotely. Most of these tools require an SSH connection that can access the application installation directly. In our case, this means we need to configure SSH keys for the www-data user.
We will need the public key file for the user you wish to push your code from. This file is typically found at ~/.ssh/id_rsa.pub. Copy that file into the ansible-php directory.
cp ~/.ssh/id_rsa.pub ~/ansible-php/deploykey.pubWe can use the Ansible authorized_key module to easily install our public key within /var/www/.ssh/authorized_keys, which will allow the deployment tools to easily connect and access our application. The configuration only needs to know where the key is, using a lookup, and the user the key needs to be installed for (www-data in our case).
- name: Copy public key into /var/www
authorized_key: user=www-data key="{{ lookup('file', 'deploykey.pub') }}"We also need to set the www-data user’s shell, so we can actually log in. Otherwise, SSH will allow the connection, but there will be no shell presented to the user. This can be done using the user module, and setting the shell to /bin/bash (or your preferred shell).
- name: Set www-data user shell
user: name=www-data shell=/bin/bashNow, open up the playbook for editing to add in the new tasks.
nano php.ymlAdd the above tasks to your php.yml playbook; the end of the file should match the following. The additions are highlighted in red.
. . .
- name: Configure nginx
template: src=nginx.conf dest=/etc/nginx/sites-available/default
notify:
- restart php5-fpm
- restart nginx
- name: Copy public key into /var/www
authorized_key: user=www-data key="{{ lookup('file', 'deploykey.pub') }}"
- name: Set www-data user shell
user: name=www-data shell=/bin/bash
handlers:
. . .Save and run the playbook.
ansible-playbook php.yml --ask-sudo-passWhen Ansible finishes, you should be able to SSH in using the www-data user.
ssh www-data@your_server_ipIf you successfully log in, it’s working! You can now log back out by entering logout or pressing CTRL+D.
We won’t need to use that connection for any other steps in this tutorial, but it will be useful if you are setting up other tools, as mentioned above, or for general debugging and application maintenance as required.
Step 3 — Configuring the Firewall
In this step we will configure the firewall on the droplet to allow only connections for HTTP and SSH respectively.
Ubuntu 14.04 comes with UFW (Uncomplicated Firewall) installed by default, and Ansible supports it with the ufw module. It has a number of powerful features and has been designed to be as simple as possible. It’s perfectly suited for self-contained web servers that only need a couple of ports open. In our case, we want port 80 (HTTP) and port 22 (SSH) open. You may also want port 443 for HTTPS.
The ufw module has a number of different options which perform different tasks. The different tasks we need to perform are:
- Enable UFW and deny all incoming traffic by default.
- Open the SSH port but rate limit it to prevent brute force attacks.
- Open the HTTP port.
This can be done with the following tasks, respectively.
- name: Enable UFW
ufw: direction=incoming policy=deny state=enabled
- name: UFW limit SSH
ufw: rule=limit port=ssh
- name: UFW open HTTP
ufw: rule=allow port=httpAs before, open the php.yml file for editing.
nano php.ymlAdd the above tasks to the the playbook; the end of the file should match the following.
. . .
- name: Copy public key into /var/www
authorized_key: user=www-data key="{{ lookup('file', 'deploykey.pub') }}"
- name: Set www-data user shell
user: name=www-data shell=/bin/bash
- name: Enable UFW
ufw: direction=incoming policy=deny state=enabled
- name: UFW limit SSH
ufw: rule=limit port=ssh
- name: UFW open HTTP
ufw: rule=allow port=http
handlers:
. . .Save and run the playbook.
ansible-playbook php.yml --ask-sudo-passWhen that has successfully completed, you should still be able to connect via SSH (using Ansible) or HTTP to your server; other ports will now be blocked.
You can verify the status of UFW at any time by running this command:
ansible php --sudo --ask-sudo-pass -m shell -a "ufw status verbose"Breaking down the Ansible command above:
ansible: Run a raw Ansible task, without a playbook.php: Run the task against the hosts in this group.--sudo: Run the command assudo.--ask-sudo-pass: Prompt for thesudopassword.-m shell: Run theshellmodule.-a "ufw status verbose": The options to be passed into the module. Because it is ashellcommand, we pass the raw command (i.e.ufw status verbose) straight in without anykey=valueoptions.
It should return something like this.
your_server_ip | success | rc=0 >>
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip
To Action From
-- ------ ----
22 LIMIT IN Anywhere
80 ALLOW IN Anywhere
22 (v6) LIMIT IN Anywhere (v6)
80 (v6) ALLOW IN Anywhere (v6)Step 4 — Installing the MySQL Packages
In this step we will set up a MySQL database for our application to use.
The first step is to ensure that MySQL is installed on our server. This can be easily achieved by adding the required packages to the install packages task at the top of our playbook. The packages we need are mysql-server, mysql-client, and php5-mysql. We will also need python-mysqldb so Ansible can communicate with MySQL.
As we are adding packages, we need to restart nginx and php5-fpm to ensure the new packages are usable by the application. In this case, we need MySQL to be available to PHP, so it can connect to the database.
One of the fantastic things about Ansible is that you can modify any of the tasks and re-run your playbook and the changes will be applied. This includes lists of options, like we have with the apt task.
As before, open the php.yml file for editing.
nano php.ymlFind the install packages task, and update it to include the packages above:
. . .
- name: install packages
apt: name={{ item }} update_cache=yes state=latest
with_items:
- git
- mcrypt
- nginx
- php5-cli
- php5-curl
- php5-fpm
- php5-intl
- php5-json
- php5-mcrypt
- php5-sqlite
- sqlite3
- mysql-server
- mysql-client
- php5-mysql
- python-mysqldb
notify:
- restart php5-fpm
- restart nginx
. . .Save and run the playbook:
ansible-playbook php.yml --ask-sudo-passStep 5 — Setting up the MySQL Database
In this step we will create a MySQL database for our application.
Ansible can talk directly to MySQL using the mysql_-prefaced modules (e.g. mysql_db, mysql_user). The mysql_db module provides a way to ensure a database with a specific name exists.
We can use a task that looks like this:
- name: Create MySQL DB
mysql_db: name=laravel state=presentWe also need a valid user account, with a known password, to allow our application to connect to the database successfully. One approach to this is to generate a password locally and save it in our Ansible playbook, but that is insecure and there is a better way. We will generate the password, using Ansible, on the server itself and use it directly where it is needed.
To generate a password, we will use the makepasswd command line tool, and ask for a 32-character password. Because makepasswd isn’t default on Ubuntu, we will need to add that to the packages list too.
We will also tell Ansible to remember the output of the command (i.e. the password), so we can use it later in our playbook. However, because Ansible doesn’t know when it’s already run a shell command before, we also need to configure a file to check for; if the file exists, Ansible assumes the command has already been run so it won’t run it again.
The task looks like this:
- name: Generate DB password
shell: makepasswd --chars=32
args:
creates: /var/www/laravel/.dbpw
register: dbpwdNext, we need to create the actual MySQL database user with the password we specified. This is done using the mysql_user module, and we can use the stdout option on the variable we defined during the password generation task to get the raw output of the shell command, like this: dbpwd.stdout.
The mysql_user command accepts the name of the user and the privileges required. In our case, we want to create a user called laravel and give them full privileges on the laravel table. We also need to tell the task to only run when the dbpwd variable has changed, which will only be when the password generation task is run.
The task should look like this:
- name: Create MySQL User
mysql_user: name=laravel password={{ dbpwd.stdout }} priv=laravel.*:ALL state=present
when: dbpwd.changedPutting this together, open the php.yml file for editing, so we can add in the above tasks.
nano php.ymlFirstly, find the install packages task, and update it to include the makepasswd package.
. . .
- name: install packages
apt: name={{ item }} update_cache=yes state=latest
with_items:
- git
- mcrypt
- nginx
- php5-cli
- php5-curl
- php5-fpm
- php5-intl
- php5-json
- php5-mcrypt
- php5-sqlite
- sqlite3
- mysql-server
- mysql-client
- php5-mysql
- python-mysqldb
- makepasswd
notify:
- restart php5-fpm
- restart nginx
. . .Then, add the password generation, MySQL database creation, and user creation tasks at the bottom.
. . .
- name: UFW limit SSH
ufw: rule=limit port=ssh
- name: UFW open HTTP
ufw: rule=allow port=http
- name: Create MySQL DB
mysql_db: name=laravel state=present
- name: Generate DB password
shell: makepasswd --chars=32
args:
creates: /var/www/laravel/.dbpw
register: dbpwd
- name: Create MySQL User
mysql_user: name=laravel password={{ dbpwd.stdout }} priv=laravel.*:ALL state=present
when: dbpwd.changed
handlers:
. . .Do not run the playbook yet! You may have noticed that although we have created the MySQL user and database, we haven’t done anything with the password. We will cover that in the next step. When using shell tasks within Ansible, it is always important to remember to complete the entire workflow that deals with the output/results of the task before running it to avoid having to manually log in and reset the state.
Step 6 — Configuring the PHP Application for the Database
In this step we will save the MySQL database password into the .env file for the application.
Like we did in the last tutorial, we will update the .env file to include our newly created database credentials. By default Laravel’s .env file contains these lines:
DB_HOST=localhost
DB_DATABASE=homestead
DB_USERNAME=homestead
DB_PASSWORD=secretWe can leave the DB_HOST line as-is, but will update the other three using the following tasks, which are very similar to the tasks we used in the previous tutorial to set APP_ENV and APP_DEBUG.
- name: set DB_DATABASE
lineinfile: dest=/var/www/laravel/.env regexp='^DB_DATABASE=' line=DB_DATABASE=laravel
- name: set DB_USERNAME
lineinfile: dest=/var/www/laravel/.env regexp='^DB_USERNAME=' line=DB_USERNAME=laravel
- name: set DB_PASSWORD
lineinfile: dest=/var/www/laravel/.env regexp='^DB_PASSWORD=' line=DB_PASSWORD={{ dbpwd.stdout }}
when: dbpwd.changedAs we did with the MySQL user creation task, we have used the generated password variable (dbpwd.stdout) to populate the file with the password, and have added the when option to ensure it is only run when dbpwd has changed.
Now, because the .env file already existed before we added our password generation task, we will need to save the password to another file so then generation task can look for it’s existence (which we already set up within the task). We will also use the sudo and sudo_user options to tell Ansible to create the file as the www-data user.
- name: Save dbpw file
lineinfile: dest=/var/www/laravel/.dbpw line="{{ dbpwd.stdout }}" create=yes state=present
sudo: yes
sudo_user: www-data
when: dbpwd.changedOpen the php.yml file for editing.
nano php.ymlAdd the above tasks to the the playbook; the end of the file should match the following.
. . .
- name: Create MySQL User
mysql_user: name=laravel password={{ dbpwd.stdout }} priv=laravel.*:ALL state=present
when: dbpwd.changed
- name: set DB_DATABASE
lineinfile: dest=/var/www/laravel/.env regexp='^DB_DATABASE=' line=DB_DATABASE=laravel
- name: set DB_USERNAME
lineinfile: dest=/var/www/laravel/.env regexp='^DB_USERNAME=' line=DB_USERNAME=laravel
- name: set DB_PASSWORD
lineinfile: dest=/var/www/laravel/.env regexp='^DB_PASSWORD=' line=DB_PASSWORD={{ dbpwd.stdout }}
when: dbpwd.changed
- name: Save dbpw file
lineinfile: dest=/var/www/laravel/.dbpw line="{{ dbpwd.stdout }}" create=yes state=present
sudo: yes
sudo_user: www-data
when: dbpwd.changed
handlers:
. . .Again, do not run the playbook yet! We have one more step to complete before we can run the playbook.
Step 7 — Migrating the Database
In this step, we will run the database migrations to set up the database tables.
In Laravel, this is done by running the migrate command (i.e. php artisan migrate --force) within the Laravel directory. Note that we have added the --force flag because the production environment requires it.
The Ansible task to perform this looks like this.
- name: Run artisan migrate
shell: php artisan migrate --force
when: dbpwd.changedNow it is time to update our playbook. Open the php.yml file for editing.
nano php.ymlAdd the above tasks to the the playbook; the end of the file should match the following.
. . .
- name: Save dbpw file
lineinfile: dest=/var/www/laravel/.dbpw line="{{ dbpwd.stdout }}" create=yes state=present
sudo: yes
sudo_user: www-data
when: dbpwd.changed
- name: Run artisan migrate
shell: php /var/www/laravel/artisan migrate --force
sudo: yes
sudo_user: www-data
when: dbpwd.changed
handlers:
. . .Finally, we can save and run the playbook.
ansible-playbook php.yml --ask-sudo-passWhen that finishes executing, refresh the page in your browser and you should see a message that says:
Queue: NO
Cron: NOThis means the database is set up correctly and working as expected.
Step 8 — Configuring cron Tasks
In this step, we will set up any cron tasks that need to be configured.
Cron tasks are commands that run on a set schedule and can be used to perform any number of tasks for your application. They are often used for performing maintenance tasks or sending out email activity updates — essentially anything that needs to be done periodically without a user starting it manually. Cron schedules can run as frequently as every minute, or as infrequently as you require.
Laravel comes by default with an Artisan command called schedule:run, which is designed to be run every minute and executes the defined scheduled tasks within the application. This means we only need to add a single cron task, if our application takes advangate of this feature.
Ansible has a cron module, which allows you to add cron tasks easily. It has a number of different options that translate directly into the different options you can configure via cron:
job: The command to execute. Required if state=present.minute,hour,day,month, andweekday: The minute, hour, day, month, or day of the week when the job should run, respectively.special_time(reboot,yearly,annually,monthly,weekly,daily,hourly): Special time specification nickname.
By default, it will create a task that runs every minute, which is what we want. This means the task we want looks like this:
- name: Laravel Scheduler
cron: >
job="run-one php /var/www/laravel/artisan schedule:run 1>> /dev/null 2>&1"
state=present
user=www-data
name="php artisan schedule:run"The run-one command is a small helper in Ubuntu that ensures the command is only being run once. This means that if a previous schedule:run command is still running, it won’t be run again. This is helpful to avoid situations where a cron task becomes locked in a loop, and over time more and more instances of the same task are started until the server runs out of resources.
As before, open the php.yml file for editing.
nano php.ymlAdd the above task to the the playbook; the end of the file should match the following.
. . .
- name: Run artisan migrate
shell: php /var/www/laravel/artisan migrate --force
sudo: yes
sudo_user: www-data
when: dbpwd.changed
- name: Laravel Scheduler
cron: >
job="run-one php /var/www/laravel/artisan schedule:run 1>> /dev/null 2>&1"
state=present
user=www-data
name="php artisan schedule:run"
handlers:
. . .Save and run the playbook:
ansible-playbook php.yml --ask-sudo-passNow, refresh the page in your browser. In a minute, it will update to look like this.
Queue: NO
Cron: YESThis means that the cron is working in the background correctly. As part of the example application, there is a cron job that is running every minute updating a status entry in the database so the application knows it is running.
Step 9 — Configuring the Queue Daemon
In this step we will configure the queue daemon worker for Laravel.
Like the schedule:run Artisan command from step 8, Laravel also comes with a queue worker that can be started with the queue:work --daemon Artisan command. We will set that up now, so you can take advantage of it in your application.
Queue workers are similar to cron jobs in that they run tasks in the background. The difference is that the application pushes jobs into the queue, either via actions performed by the user, or from tasks scheduled through a cron job. Queue tasks are executed by the worker one at a time, and will be processed on-demand when they are found in the queue. They are commonly used for tasks that take time to execute, such as sending emails or making API calls to external services.
Unlike the schedule:run command, this isn’t a command that needs to be run every minute. Instead it needs to be running as a daemon in the background constantly. A common way to do this is to use a third party package, like supervisord, but that method requires understanding how to configure and manage said system. There is a much simpler way to do it using cron and the run-one command.
We will create a cron entry to start the queue worker daemon, and use run-one to run it. This means that cron will start the process the first time it runs, and any subsequent cron runs will be ignored by run-one while the worker is running. As soon as the worker stops, run-one will allow the command to run again, and the queue worker will start again. It is an incredibly simple and easy to use method that saves you from needing to learn how to configure and use another tool.
With all of that in mind, we will create another cron task to run our queue worker.
- name: Laravel Queue Worker
cron: >
job="run-one php /var/www/laravel/artisan queue:work --daemon --sleep=30 --delay=60 --tries=3 1>> /dev/null 2>&1"
state=present
user=www-data
name="Laravel Queue Worker"As before, open the php.yml file for editing.
nano php.ymlAdd the above task to the the playbook; the end of the file should match the following:
. . .
- name: Laravel Scheduler
cron: >
job="run-one php /var/www/laravel/artisan schedule:run 1>> /dev/null 2>&1"
state=present
user=www-data
name="php artisan schedule:run"
- name: Laravel Queue Worker
cron: >
job="run-one php /var/www/laravel/artisan queue:work --daemon --sleep=30 --delay=60 --tries=3 1>> /dev/null 2>&1"
state=present
user=www-data
name="Laravel Queue Worker"
handlers:
. . .Save and run the playbook:
ansible-playbook php.yml --ask-sudo-passLike before, refresh the page in your browser. After a minute, it will update to look like this:
Queue: YES
Cron: YESThis means that the queue worker is working in the background correctly. The cron job that we started in the last step is pushing a job onto the queue. This job updates the database when it is run, to show that it is working.
We now have a working example Laravel application which includes functioning cron jobs and queue workers.
Conclusion
This tutorial covered the some of the more advanced topics when using Ansible for deploying PHP applications. All of the tasks used can be easily modified to suit most PHP applications (depending on their specific requirements), and it should give you a good starting point to set up your own playbooks for your applications.
You will notice that we have not used a single SSH command as part of this tutorial (apart from checking the www-data user login), and everything — including the MySQL user password — has been set up automatically. After following this tutorial, your application is ready to go and supports tools to push code updates.