You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Terraform Google Cloud with Rails 8 App and Kamal v2
2
+
2
3
This repository contains a Rails 8 app with Terraform files for deploying the app on Google Cloud. The app uses Kamal v2 for deployment.
3
4
4
-
Why use Rails 8 with Terraform, Google Cloud, and Kamal v2?
5
+
## Consulting Help Available
6
+
* Would like assistance with using Kamal, Docker, Terraform, or Google Cloud?
7
+
* Do you have concerns about the performance, security, and observability of your current deployments?
8
+
* Are you interested in optimizing your deployment process to get Heroku features without the high costs?
9
+
10
+
If so, please [email me](mailto:[email protected]) or [book a time](https://meetings.hubspot.com/justingordon/30-minute-consultation).
11
+
*[Click to join **React + Rails Slack** to chat with Justin](https://reactrails.slack.com/join/shared_invite/enQtNjY3NTczMjczNzYxLTlmYjdiZmY3MTVlMzU2YWE0OWM0MzNiZDI0MzdkZGFiZTFkYTFkOGVjODBmOWEyYWQ3MzA2NGE1YWJjNmVlMGE).
12
+
* Check out [ShakaCode's Infrastructure Optimization Services](https://www.shakacode.com/services/it-infrastructure-optimization/).
13
+
14
+
## Kamal Basics
15
+
16
+
In order to use Kamal, you should try to understand what's going on from first principles.
17
+
18
+
Kamal is a CLI that uses configuration files to orchestrate commands for Docker deployment on remote machines.
19
+
20
+
Docs are nice. But lots are not in the docs. That's OK for 2 reasons:
21
+
1. Kamal gives lots of output to the command line on what's running
22
+
2. You have the source code, and you can ask AI for help.
23
+
24
+
## Why use Rails 8 with Terraform, Google Cloud, and Kamal v2?
25
+
5
26
1.**Infrastructure as Code**: Terraform allows you to define your infrastructure in code, making it easier to manage and scale. By using Terraform, you don't have to configure anything in the Google Cloud Console manually.
6
27
2.**Consistent Deployments**: With Terraform, you can ensure that your infrastructure is consistent across all environments. This helps in reducing errors and ensuring that your app runs smoothly.
7
28
3. Scripts to stand-up and tear-down the infrastructure: The Terraform files in this repository include scripts to create and destroy the infrastructure. This makes it easy to spin up a new environment for testing and tear it down when you're done. Don't pay for resources you're not using!
8
29
4.**Kamal v2**: Kamal v2 is a lightweight deployment tool that makes it easy to deploy Rails apps to Google Cloud. It handles the deployment process for you, so you don't have to worry about setting up Kubernetes clusters or managing containers.
9
30
5.**Rails 8**: Rails 8 is the latest version of the popular Ruby on Rails framework. It comes with many new features and improvements that make it easier to build web applications.
10
31
6.**Google Cloud**: Google Cloud is a powerful cloud platform that offers a wide range of services for building and deploying applications. By using Google Cloud, you can take advantage of its scalability, reliability, and security features.
11
32
12
-
13
33
## Requirements
14
34
15
35
1.[Google Cloud SDK](https://cloud.google.com/sdk/docs/install) with a gcloud account.
@@ -25,22 +45,27 @@ Why use Rails 8 with Terraform, Google Cloud, and Kamal v2?
Or just make a brand new one for development and test with `rails credentials:edit`.
28
-
29
-
2. Install the required gems:
48
+
3. Install the required gems:
30
49
```bash
31
50
bundle install
32
51
```
33
-
3. Edit the `terraform-gcloud/variables.tf` file with your project details.
34
-
4. Edit the `config/deploy.yml` file with your domain name (currently set to `kamal.shakacode.com`)
35
-
5.
36
-
37
-
38
-
# Secrets and Credentials
52
+
4. Edit the `terraform-gcloud/variables.tf` file with your project details. You need to create a Google Cloud "project" for this demo.
53
+
5. Edit the `config/deploy.yml` file:
54
+
1. Set the `proxy.host` domain name (currently set to `gcp.kamaltutorial.com`)
55
+
2. Change the `registry.username`
56
+
3. Change the `ssh.user` to your username.
57
+
58
+
## Rails 8 Defaults
59
+
This Rails 8 example app differs as little as possible from the default Rails 8 app. The main differences are:
60
+
1. Terraform setup in the `terraform-gcloud` directory. Terraform is super nice because you can follow the example with minimal work to get your Rails app running on Google Cloud. For a non-tutorial application, you'd put the Terraform files in a separate git repo.
61
+
2. The docker and Kamal setup has minimal changes.
62
+
63
+
## Secrets and Credentials
39
64
Note:
40
65
1. I originally created the example to work with 1Password. However, given that a GCP account is required, I decided to use the GCP Secret Manager.
41
66
2. To demonstrate best practices for handling secrets, we will NOT use rails credentials for production secrets. Instead, we will use the Google Cloud Secret Manager. Rails credentials are great for non-production environments.
42
67
43
-
## Google Cloud Secret Manager
68
+
###Google Cloud Secret Manager
44
69
45
70
1. Open the [Google Cloud Secret Manager](https://console.cloud.google.com/security/secret-manager).
46
71
2. Ensure that you have the correct project selected.
@@ -50,38 +75,158 @@ Note:
50
75
-`DB_PASSWORD`: The password for the database, as you like.
51
76
-`SECRET_KEY_BASE`: Generate with `rails secret`.
52
77
53
-
# Database Setup and Migrations
78
+
### `deploy.yml`
79
+
Note that the `deploy.yml` already has:
80
+
```yaml
81
+
env:
82
+
secret:
83
+
- DB_PASSWORD
84
+
- SECRET_KEY_BASE
85
+
```
86
+
87
+
### `.kamal/secrets`
88
+
The `.kamal/secrets` already has this code. Conveniently, you don't need to duplicate your GCP PROJECT_ID.
Note that the `deploy.yml` file ensures that the DB_HOST and DB_PASSWORD are set in the environment. The DB_HOST is set to a seemingly magic value of
97
+
```yaml
98
+
DB_HOST: 172.18.0.1
99
+
```
100
+
101
+
This is the value that Docker uses to refer to the host machine from within a Docker container. This is because the database is referenced as localhost on the host machine, not localhost in the Docker container. This way, we don't need an IP address for the database. The terraform script sets up the database to work like this by installing `cloud_sql_proxy` and running it.
102
+
103
+
104
+
The `config/database.yml` file already has the following code, which needs to correspond to the secrets and the setup of the databases in `terraform-gcloud/main.tf`:
The database is automatically created and migrated when the Rails app is deployed. This is done via the [bin/docker-entrypoint.sh](bin/docker-entrypoint.sh) script which calls `rails db:prepare`.
55
130
56
131
Note, the initial deployment will fail because the database schema needs to be created. This is expected. Just run `bundle exec kamal deploy` again after the initial setup.
57
132
58
-
# Deployment
59
-
60
-
1. To create the infrastructure on Google Cloud, run the following command:
61
-
```bash
62
-
./terraform-gcloud/bin/stand-up
63
-
```
64
-
2. Notice the outputted IP address. Add an A record to your domain name pointing to this IP address.
65
-
3. Deploy the Rails app using Kamal v2:
66
-
```bash
67
-
bundle exec kamal setup
68
-
```
69
-
Notice an error message. This will happen because the schema needs to created the first time.
70
-
4. Deploy the Rails app using Kamal v2:
71
-
```bash
72
-
bundle exec kamal deploy
73
-
```
74
-
75
-
4. Visit your domain name in the browser to see your Rails app running on Google Cloud!
76
-
77
-
# Tear Down
78
-
79
-
1. To destroy the infrastructure on Google Cloud, run the following command:
80
-
```bash
81
-
./terraform-gcloud/bin/tear-down
82
-
```
83
-
That's it!
84
-
85
-
86
-
87
-
# Troubleshooting
133
+
## Automated Deployment
134
+
Run `bin/terraform-gcloud/bin/stand-up` to create the infrastructure on Google Cloud and deploy the Rails app using Kamal v2.
135
+
136
+
This script has some useful features:
137
+
138
+
1. It creates the infrastructure on Google Cloud using Terraform.
139
+
2. It updates the deploy.yml config file to reflect the IP address of the server and makes a longer deployment timeout.
140
+
3. You get a chance to add an A record to your domain name pointing to the IP address.
141
+
4. It deploys the Rails app using Kamal v2.
142
+
5. Visit your domain name in the browser to see your Rails app running on Google Cloud!
143
+
144
+
## Tear Down
145
+
146
+
When you're done, run `bin/terraform-gcloud/bin/tear-down` to destroy the infrastructure on Google Cloud (and save any costs!)
147
+
148
+
The `tear-down` script has some useful features:
149
+
1. Ensures calling `kamal app stop` or else terraform cannot destroy the database.
150
+
2. Call `terraform destroy` to destroy the infrastructure on Google Cloud.
151
+
152
+
## Step by Step
153
+
To get a sense of the basics of Terraform and Kamal v2, follow these steps.
154
+
155
+
### Terraform Setup
156
+
First, ensure that you can run `terraform` commands to create the infrastructure on Google Cloud.
157
+
158
+
1. Install the Google Cloud SDK and Terraform.
159
+
2. Run terraform commands from `cd terraform-gcloud`.
160
+
2. Update the `terraform-gcloud/variables.tf` file with your project details as described above.
161
+
3. Run `terraform init` in the `terraform-gcloud` directory to initialize the Terraform configuration.
162
+
4. Run `terraform plan` to see the changes that Terraform will make to your infrastructure.
163
+
5. Run `terraform apply` to create the infrastructure on Google Cloud. This takes about 10 minutes mainly due to provisioning the database. Note that the output will include the IP address of the server. You need this for 2 reasons:
164
+
1. To add an A record to your domain name.
165
+
2. To update the `config/deploy.yml` file with the IP address for your server.
166
+
6. Run `terraform destroy` to tear down the infrastructure when you're done (after practicing the Kamal deployment)
167
+
168
+
### Kamal v2 Deployment
169
+
Next, ensure that you can deploy the Rails app using Kamal v2.
170
+
171
+
1. Run `./bin/kamal setup` to set up the Kamal v2 configuration. Notice the error that the health checks failed. This is expected because the database needs to be created and migrated.
172
+
2. Unless you change the `deploy.yml` file to have a longer `deploy_timeout`, you'll need to run `./bin/kamal deploy` a second time, because the database needs to be created and migrated. The `stand-up` script does this for you.
173
+
3. Visit your domain name in the browser to see your Rails app running on Google Cloud!
174
+
175
+
## Troubleshooting
176
+
If you encounter any issues during the deployment process, here are some common troubleshooting steps.
177
+
178
+
### Execution Flow
179
+
First, it's important to understand the execution context of when running commands.
180
+
181
+
1. **Your Local Machine (or CI Machine):**
182
+
* Runs commands like kamal deploy, which connects to the host machine via SSH.
183
+
2. **Host Machine:**
184
+
* Receives commands from Kamal and executes them, managing the Docker runtime environment.
185
+
* Temporarily hosts deployment files (e.g., hook scripts) and runs them within Docker containers.
186
+
* `ssh user@host` to get a shell on the host machine.
187
+
3. **Docker Machine (Containers):**
188
+
* The host machine runs Docker containers for the Rails app and the Kamal proxy.
189
+
* The app runs here. Commands like bin/rails db:migrate execute inside these containers.
190
+
* `docker ps` to see the running containers.
191
+
* `docker logs CONTAINER_ID` to see the logs of a container.
192
+
* `docker exec -it CONTAINER_ID bash` to get a shell in a container.
193
+
194
+
### Troubleshooting Steps
195
+
1. First, read the console messages very carefully and look for the first error message. This is often the most important clue. If there's a health check timeout, it might be due to the failure to run migrations quickly enough, and then you simply need to run `./bin/kamal deploy` again.
196
+
2. Check the logs for the Rails app and the Kamal proxy to see if there are any error messages. You can do this with the command `./bin/kamal logs`.
197
+
3. If there is trouble with the database, then you won't get far because your default ENTRYPOINT `bin/docker-entrypoint` will fail. You need to first find the CONTAINER_ID of the failing container. You can do this by:
198
+
1. ssh to the host machine, like `ssh <username>@<ip_address>`.
199
+
2. Run `docker ps` and export a value for CONTAINER_ID. Then you can run `docker logs $CONTAINER_ID` to see the logs.
200
+
3. You can run `docker run -it --entrypoint bash $CONTAINER_ID` to get a shell and then run `bin/docker-entrypoint` to see what's going on. This skips your default ENTRYPOINT.
201
+
202
+
### Debugging Tips with AI Tools
203
+
204
+
Next, use AI tools to help you debug. A prompt like this is very helpful. Substitute your host IP address and the Rails container ID.
205
+
206
+
```
207
+
I'm using Kamal v2. Double check you are not giving me answers for Kamal v1
208
+
209
+
When you give me commands, tell me which execution context: Local machine, host machine, or docker container.
210
+
211
+
The remote host IP is 34.122.124.21.
212
+
213
+
The rails app docker container id is f41ea810b98f
214
+
```
215
+
216
+
To get a good understanding of what's going on, you can run the following commands:
217
+
218
+
```
219
+
Walk me through the output, one command at a time for the following output of kamal v2.
220
+
Don't analyze everything. Go one command at a time.
221
+
Only analyze the "Running" lines, one at a time.
222
+
223
+
<THENPASTECOMMANDANDOUTPUT>
224
+
```
225
+
226
+
## Unaddressed concerns
227
+
1. Machine monitoring.
228
+
* What happens when disk runs out of space?
229
+
* What happens if memory maxes out?
230
+
* What happens if CPU maxes out?
231
+
* No Auto-Scaling
232
+
2. Machine must have about twice as much memory as new app needs to run with old app during deployment. Why pay for all that extra memory when not needed?
0 commit comments