Django Squashing Database Migrations Files

Migrations are Django's way of propagating changes we make to our models (adding a field, deleting a model, etc.) into our database schema.

Migrations in Django propagate model changes (like adding a field) to our database schema. The key commands are:

• migrate: Applies and unapplied migrations.
• makemigrations: Creates new migration files based on model changes.
• sqlmigrate: Shows the SQL statements for a migration.
• showmigrations: Lists migrations and their status.

In a typical Django project, Migration files are stored in each app's "migrations" directory and should be shared across development, staging, and production environments. Think of migrations as a version control system for our database schema. makemigrations is responsible for packaging up our model changes into individual migration files - analogous to commits - and migrate is responsible for applying those to our database.

What if two migrations have the same number while collaborative development on the same App?

Migrations specify which other migrations they depend on - including earlier migrations in the same app - in the file, so it’s possible to detect when two new migrations for the same app aren’t ordered. By the way, the numbers in the migrations are only for reference to the developer. Django specifies the dependencies in the migration files. Using those dependencies, it can detect the sequence of migrations.

As of now, we might have the idea of the migrations in the Django Framework, so we can discuss about the Squashing Migrations files into one file.

Squashing migrations in Django

Squashing migrations in Django is a process that combines multiple migrations into a single file. Squashing in Django reduces multiple migrations into one by sequencing and optimizing operations. Django intelligently manages transitions, allowing new installs to use the squashed migration while existing systems finish applying old ones. The process ensures smooth updates without disrupting production environments, with old files removed after all systems are updated.

Syntax:

python manage.py squashmigrations app_name 0001_initial 0010_last_migration

Here:

• app_name - refers to a Django app in our project.
• 0001_initial - refers to the start of the migrations file.
• 0010_last_migration - refers to the end of the migrations file.

Step to squash the existing migrations:

To squash migrations in a Django project, we can follow these steps:

Step 1: Ensure the Migrations Are Up-to-Date

First, make sure all the migrations have been applied:

python manage.py migrate

This ensures that the database schema is in sync with all existing migrations.

Step 2: Squash Migrations

We can use the `squashmigrations` command to combine the migrations. Replace `app_name` with the name of the app and specify the migration range we want to squash:

For example, if we want to squash all migrations in the `myapp` app:

//Syntax of the code:
python manage.py squashmigrations app_name start_migration end_migration

// Example:
python manage.py squashmigrations myapp 0001_initial 0100_migration_2024

Step 3: Review the Squashed Migration

Django will create a new migration file. Open this file in the text editor and review it to ensure that it correctly represents the schema changes. Make sure any custom operations (like `RunPython` or `RunSQL`) are properly handled.

Step 4: Apply the Squashed Migration

Apply the squashed migration to our database:

python manage.py migrate

Step 5: Commit the Changes

Commit the new squashed migration and the previous migration files to our version control system:

git add .
git commit -m "Squashed migrations for myapp"

Step 6: Clean Up Old Migrations (Optional)

After ensuring all environments are up-to-date, we can remove the old migration files:

rm -rf myapp/migrations/old_migration_files.py

We can run the above step, only when we are confident enough with new squashed migration, otherwise it will become a blunder in the code.

Step 7: Release and Deploy

Finally, release and deploy the changes. Ensure that all environments are updated using the new squashed migration before removing the old files in a subsequent release.

Squashing necessary for several reasons:

• Performance Improvement: Over time, as we develop a Django project, the number of migrations can grow significantly. Each migration is a separate database operation, and running dozens or hundreds of them can slow down deployment times. Squashing condenses these operations, reducing the overhead and speeding up the migration process.
• Codebase Cleanliness: A large number of migration files can clutter the codebase, making it harder to manage and understand. Squashing simplifies the migration history, making it easier for developers to navigate and maintain the project.
• Avoiding Redundancy: In long-lived projects, there can be many migrations that add and then remove fields or models. These operations become redundant over time. Squashing migrations can eliminate this redundancy, ensuring that only the necessary schema changes are preserved.
• Reducing Merge Conflicts: In teams, multiple developers working on different migrations can lead to merge conflicts. Squashing reduces the number of migrations and, by extension, the likelihood of conflicts, simplifying collaborative development.

While squashing can be beneficial, it should be done with care, particularly in projects with complex migration histories, to avoid data loss or other issues. Take care of the CircularDependencyErrors, which occur due to the interdependencies (Foreign-key relations) among the models.

Project Demonstration of Squashing migrations:

• In this example, we will have a small project to demonstrate the squashing of migrations.
• First, Inside out project, we will create a demo app. In this app, we will have the definition of models, views and other files.
• Then, in the models.py file, lets write the model. After the final iteration, we have the following file as final code in model.py.

models.py

from django.db import models

class Author(models.Model):
    first_name = models.CharField(verbose_name="first_name")
    middle_name = models.CharField(verbose_name="middle_name")
    last_name = models.CharField(verbose_name="last_name")
    
class Publisher(models.Model):
    name = models.CharField(verbose_name="publisher")
    location = models.CharField(verbose_name="publisher head quarter")

class Book(models.Model):
    title = models.CharField(verbose_name="book_name")
    # altered the fields for author and publisher:
    author = models.ForeignKey(Author, on_delete=models.CASCADE)
    publisher = models.ForeignKey(Publisher, on_delete=models.CASCADE)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)
    # changes made in first iterations:
    price = models.PositiveIntegerField(verbose_name="price", null=True)
    # changes made in second iterations:
    length = models.PositiveIntegerField(verbose_name="Book length", null=True)
    width = models.PositiveIntegerField(verbose_name="Book width", null=True) 

On the basis of the above models, when we run makemigrations on each significant change, the migrations folder will contain a lot of migration files.

The below screenshot demonstrate the migrations folder is containing a lot of files. Also, in the carousel, you can see the changes made.

After making these much changes and migrating on each change we can see in the below screenshot, that numbers of files in migrations folder is increased significantly.

This is a demonstration project, even small changes were migrated to generate migration files, but in real world examples, the number of applications in the project can be significant and so the changes. This clear that migration folders can have more than 100 migrations file for each application.

The code to squash the migrations of the changes made in the model can be seen below:

python manage.py squashmigrations demo 0001_initial 0005_publisher

After executing the above command, the terminal will ask for yes/no for squashing. If you proceed with yes, then the result will be as shown below:

After squashing all the migrations, run the server and test your application. DO NOT Forget to see all the functions are working properly or not. If the app is working fine then you can delete the previous migrations files.

Remember: Do not delete the squashed migration file.

Hence, we can see the benefits of space saving and data saving while uploading these squashed migrations rather than more than 100 migration files.

Conclusion

By following the steps outlined—ensuring all migrations are up-to-date, squashing migrations, reviewing the squashed file, and cleaning up old files—we can significantly improve the performance and cleanliness of our codebase. However, it's important to exercise caution during this process, as improper handling could lead to issues such as CircularDependencyErrors or loss of critical data.

Squashing migrations not only enhances deployment speed but also reduces redundancy and minimizes the risk of merge conflicts in collaborative development. While this process is powerful, it should be done with care, particularly in long-lived projects with complex dependencies.