Luce Carter for MongoDB

Posted on Apr 8 • Edited on Apr 11

Getting Started With Queryable Encryption in the MongoDB C# Driver

#security #dotnet #encryption #webdev

With the release of MongoDB 7.0 in August 2023, we introduced a feature called Queryable Encryption, the first of its kind. With queryable encryption, your data is encrypted, even at rest, with the server unable to read it either but still able to execute queries against it. You can specify what fields to encrypt so you can encrypt as much or as little of your document as you need.

The great news is, not only is this available for all tiers, but it is supported in our C# driver too!

In this tutorial, we are going to add queryable encryption to a healthcare application, ensuring that private information, such as social security number (SSN) and date of birth, is encrypted.

Prerequisites

To follow along with this tutorial, you will need a few things in place:

.NET 9.
A forked and cloned copy of the GitHub repo. Ensure you are on the start-qe branch as we will build off this starting point.
Any MongoDB cluster on version 7.0 or later. I will be using a free-forever M0 tier cluster on Atlas.

Note: If you want to see the final code, there is a branch on the GitHub repo called with-queryable-encryption which is a working sample. You will just need to follow the next section on adding the automatic encryption library to run it locally.

Adding the automatic encryption shared library

The first thing to do is add the automatic encryption shared library to the project. You can find this in our download center. Be sure to select crypt_shared from the package dropdown box.

Download the correct version for your platform and then unzip it to the root of the project. You should end up with a folder named something like mongo_crypt_shared_v1-macos-arm64-enterprise-8.0.4. The final folder name will change depending on your platform and version downloaded but the contents should be the same. Next, we will get the path to the mongo_crypt file that we will add to appsettings.

On Windows:
Copy the path to the .dll file found in the bin folder.

On MacOS:
Copy the path to the .dylib file in the lib folder.

Updating appsettings

Now you have the path to the mongo_crypt file on your clipboard, it is time to update appsettings.json and appsettings.Development.json.

First, update the placeholder for the CryptSharedLibPath with the file path you copied to the clipboard.

After that, update the MongoDBConnectionString value with the connection string for your MongoDB cluster.

Adding QueryableEncryptionHelpers.cs

Now, we are going to add a new class in the Services folder. This class will act as a helper with methods for handling key management for the encryption.

Inside the Services folder, add a new class called QueryableEncryptionHelpers.cs and paste the following code:

using System.Security.Cryptography;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Encryption;
namespace EnterpriseHealthcareDotNet.Services;

public class QueryableEncryptionHelpers
{
  private readonly IConfigurationRoot _appSettings;
  public QueryableEncryptionHelpers(IConfigurationRoot appSettings)
  {
    _appSettings = appSettings;
  }

  public Dictionary<string, IReadOnlyDictionary<string, object>> 
GetKmsProviderCredentials(string kmsProviderName,
        bool generateNewLocalKey)
  {
    if(kmsProviderName == "local")
    {
      if (generateNewLocalKey)
      {
        File.Delete("customer-master-key.txt");

        // start-generate-local-key
        using var randomNumberGenerator = RandomNumberGenerator.Create();
        try
        { 
          var bytes = new byte[96];
          randomNumberGenerator.GetBytes(bytes);
          var localCustomerMasterKeyBase64 = Convert.ToBase64String(bytes);
          File.WriteAllText("customer-master-key.txt", localCustomerMasterKeyBase64);
        }
        catch (Exception e)
        {
          throw new Exception("Unable to write Customer Master Key file due to the following error: " + e.Message);
        }
    // end-generate-local-key
    }

    // start-get-local-key
    // WARNING: Do not use a local key file in a production application
   var kmsProviderCredentials = new Dictionary<string, IReadOnlyDictionary<string, object>>();
   try
   {
     var localCustomerMasterKeyBase64 = File.ReadAllText("customer-master-key.txt");
     var localCustomerMasterKeyBytes = Convert.FromBase64String(localCustomerMasterKeyBase64);

     if (localCustomerMasterKeyBytes.Length != 96)
     {
       throw new Exception("Expected the customer master key file to be 96 bytes.");
     }

     var localOptions = new Dictionary<string, object>
     {
       { "key", localCustomerMasterKeyBytes }
     };

     kmsProviderCredentials.Add("local", localOptions);
     }
      // end-get-local-key
     catch (Exception e)
     {
       throw new Exception("Unable to read the Customer Master Key due to the following error: " + e.Message);
     }
 return kmsProviderCredentials;

 }

 throw new Exception("Unrecognized value for KMS provider name \"" + kmsProviderName + "\"  encountered while retrieving KMS credentials.");
 }

public BsonDocument GetCustomerMasterKeyCredentials(string kmsProvider)
{
  if (kmsProvider == "local")
  {
    // start-kmip-local-cmk-credentials
    var customerMasterKeyCredentials = new BsonDocument();
    // end-kmip-local-cmk-credentials
    return customerMasterKeyCredentials;
  }
  else
    {
      throw new Exception("Unrecognized value for KMS provider name \"" + kmsProvider + "\"  encountered while retrieving Customer Master Key credentials.");
    }
  }

public AutoEncryptionOptions GetAutoEncryptionOptions(CollectionNamespace keyVaultNamespace,
        IReadOnlyDictionary<string, IReadOnlyDictionary<string, object>> kmsProviderCredentials)
{
  var kmsProvider = kmsProviderCredentials.Keys.First();

  // start-auto-encryption-options
  var extraOptions = new Dictionary<string, object>
  {
    { "cryptSharedLibPath", _appSettings["CryptSharedLibPath"] } 
     // Path to your Automatic Encryption Shared Library
  };

var autoEncryptionOptions = new AutoEncryptionOptions(
keyVaultNamespace,
kmsProviderCredentials,
extraOptions: extraOptions);
// end-auto-encryption-options

  return autoEncryptionOptions;
}

public ClientEncryption GetClientEncryption(IMongoClient keyVaultClient,
        CollectionNamespace keyVaultNamespace, Dictionary<string, IReadOnlyDictionary<string, object>> kmsProviderCredentials)
{
  var kmsProvider = kmsProviderCredentials.Keys.First();

  // start-client-encryption
  var clientEncryptionOptions = new ClientEncryptionOptions(
                keyVaultClient: keyVaultClient,
                keyVaultNamespace: keyVaultNamespace,
                kmsProviders: kmsProviderCredentials
  );
  var clientEncryption = new ClientEncryption(clientEncryptionOptions);
            // end-client-encryption
            return clientEncryption;
  }
}

This code uses features from both the MongoDB Driver and the additional MongoDB.Driver.Encryption package inside methods for client and master key management.

This only provides code for local keys as this is what we are using in this tutorial, but you can find examples for also handling Azure, GCP, AWS, and KMIP in the same file on the GitHub repo on the with-queryable-encryption branch.

Note: Local keys are great for development but not recommended for production! So if you are looking to deploy to production, be sure to take a look at using one of the cloud providers!

Updating MongoDBService.cs

Now we have the helpers class available to use, we can start working on updating MongoDBService.cs to call the newly available methods.

At the top of the class, add a new local variable:

private readonly QueryableEncryptionHelpers _qeHelpers;

Then, inside the constructor before the call to InitAsync, initialize the variable:

_qeHelpers = new QueryableEncryptionHelpers((IConfigurationRoot)_appSettings); 
_kmsProviderCredentials = _qeHelpers.GetKmsProviderCredentials(_kmsProviderName, generateNewLocalKey: true);

Next, add the following code to InitAsync:

var camelCaseConvention = new ConventionPack { new CamelCaseElementNameConvention() };
ConventionRegistry.Register("CamelCase", camelCaseConvention, type => true);

MongoClientSettings.Extensions.AddAutoEncryption(); // .NET/C# Driver v3.0 or later only
var clientSettings = MongoClientSettings.FromConnectionString(_uri);
        clientSettings.AutoEncryptionOptions = _qeHelpers.GetAutoEncryptionOptions(
            _keyVaultNamespace,
            _kmsProviderCredentials);
var encryptedClient = new MongoClient(clientSettings);

var keyDatabase = encryptedClient.GetDatabase(_keyVaultDatabaseName);

var encryptedFields = new BsonDocument
{
  {
    "fields", new BsonArray
    {
      new BsonDocument
        {
          { "keyId", BsonNull.Value },
          { "path", "patientRecord.ssn" },
          { "bsonType", "string" },
          { "queries", new BsonDocument("queryType", "equality") 
         }
        },
       new BsonDocument
         {
           { "keyId", BsonNull.Value },
           { "path", "dateOfBirth" },
           { "bsonType", "date" },
           { "queries", new BsonDocument("queryType", "range") }
           }
          }
        }
      };

var patientDatabase = encryptedClient.GetDatabase(_encryptedDatabaseName);

var clientEncryption = _qeHelpers.GetClientEncryption(encryptedClient,
_keyVaultNamespace,
_kmsProviderCredentials);

var customerMasterKeyCredentials = _qeHelpers.GetCustomerMasterKeyCredentials(_kmsProviderName);

if(!encryptedClient.GetDatabase(_encryptedDatabaseName).ListCollectionNames().ToList().Contains(_encryptedCollectionName))
{
  try
  {
    // start-create-encrypted-collection
    var createCollectionOptions = new CreateCollectionOptions<Patient>
    {
    EncryptedFields = encryptedFields
    };

    await clientEncryption.CreateEncryptedCollectionAsync(patientDatabase,
   _encryptedCollectionName, createCollectionOptions, _kmsProviderName, customerMasterKeyCredentials);
   // end-create-encrypted-collection
   }
   catch (Exception e)
     {
       throw new Exception("Unable to create encrypted collection due to the following error: " + e.Message);
     }
   }
   _patientsCollection = encryptedClient.GetDatabase(_encryptedDatabaseName).
GetCollection<Patient>(_encryptedCollectionName);

Let’s talk through what is happening in this code.

First, it sets up a case convention that automatically handles the difference in casing between the fields in the document when stored, and the casing used for the properties in Patient model, found in the Models folder.

Then, it sets up an encrypted client, configuring the connection string and key database and turning on auto encryption. Afterward, the encrypted fields are defined. We don’t need to encrypt every field in the document—just the ones that are private, such as the social security number (SSN) and date of birth. If this was being used in production, a healthcare provider might want to find all patients born between certain dates, to target them for specific vaccines, so the date of birth field is specifically configured to allow range queries.

This defines the encrypted fields for the collection. It is used when creating the collection for the first time and is not needed on every startup.

Next, it sets up the database, fetches the credentials, and then creates the collection if it doesn’t already exist, configuring it to encrypt the earlier defined fields.

Get all patients

Everything is in place now for configuring and setting up queryable encryption in our application, so it is time to put it into place.

Still inside MongoDBService.cs, add a new method after InitAsync:

public async Task<List<Patient>> GetPatientsAsync()
{
  if (_patientsCollection == null)
    throw new InvalidOperationException("Patients collection is not initialized");

  return await _patientsCollection.Find(_ => true).ToListAsync();
}

The repo already has pages created for interacting with patients, but we need to add the code to talk to our service class and call our newly added method.

Inside the @code block in Components/Pages/Patients.razor, add the following inside the OnInitializedAsync method: patients = await MongoDbService.GetPatientsAsync();.

This will populate the List<Patients> Patients variable with all the patients from the database, and display them on the page.

You will notice that there is an existing method available to navigate to a specific patient but that functionality is not defined in code yet, so let’s do that now.

Get patient by Id

In MongoDBService.cs, add the following method, below the method for getting all patients:

public async Task<Patient> GetPatientAsync(string id)
{
  if (_patientsCollection == null)
    throw new InvalidOperationException("Patients collection is not initialized");

  return await _patientsCollection.Find(p => p.Id == ObjectId.Parse(id)).FirstOrDefaultAsync();
}

Next, we want to call another override method in the code block of PatientDetails.razor, this time, for when the parameters update:

protected override async Task OnParametersSetAsync()
{
  if (!string.IsNullOrEmpty(Id))
  {
    patient = await MongoDbService.GetPatientAsync(Id);
  }
}

This already has the Id set up as a query parameter so when the page loads, the method will be called and the patient’s details will be fetched from the service class using that id.

Of course, there are currently no patients so if we ran the application now, we would see a message on the patients page saying there are no patients. We wouldn’t be able to access this page with a valid id. So let’s change that.

Add patients

Back in MongoDBService.cs, add the following to save new patients to the database:

public async Task AddPatientAsync(Patient patient)
{
  if (_patientsCollection == null)
    throw new InvalidOperationException("Patients collection is not initialized");

  await _patientsCollection.InsertOneAsync(patient);
}

The form for adding new patients is already defined in AddPatient.razor. we just need to call our new method from the submit method.

Inside HandleSubmit, before the call to navigate back to /patients, add await MongoDbService.AddPatientAsync(newPatient);

Updating a patient

Now we have the create and read methods from the CRUD (create, read, update, delete) operations defined, it is time to handle updating.

We are going to take advantage of the ability with the driver to replace whole documents. In MongoDBService.cs, add the following method after the previous:

public async Task UpdatePatientAsync(string id, Patient patient)
{
  if(_patientsCollection == null)
    throw new InvalidOperationException("Patients collection is not initialized");

  await _patientsCollection.ReplaceOneAsync(p => p.Id == 
  ObjectId.Parse(id), patient);
}

Now, we are actually going to change PatientDetails.razor with more changes than just calling the new method, as we want to add the ability to edit an existing patient and then call the new method to update it in the database.

Replace the whole file with the following code:

@page "/patient/{id}"
@using EnterpriseHealthcareDotNet.Models
@using EnterpriseHealthcareDotNet.Services
@using MongoDB.Bson
@inject MongoDBService MongoDbService
@inject NavigationManager NavigationManager

<h3>Patient Details</h3>

@if (patient == null)
{
    <p>Loading...</p>
}
else
{
    <div>
        <p><strong>Name:</strong> <input type="text" @bind="patient.PatientName" readonly="@(!isEditMode)" /></p>
        <p><strong>Date of Birth:</strong> <input type="date" @bind="patient.DateOfBirth" readonly="@(!isEditMode)" /></p>
        <p><strong>Health Conditions: </strong></p>
        <table class="table table-hover table-responsive-sm">
            <thead>
            <tr>
                <th>Name</th>
                <th>Date</th>
                <th>Status</th>
            </tr>
            </thead>
            <tbody>
            @if(patient.PatientRecord.HealthConditions == null || !patient.PatientRecord.HealthConditions.Any())
            {
                <tr>
                    <td colspan="3">No health conditions found.</td>
                </tr>
            }
            else
            {
                @foreach (var condition in patient.PatientRecord.HealthConditions)
                {
                    <tr>
                        <td><input type="text" @bind="condition.Name" readonly="@(!isEditMode)" /></td>
                        <td><input type="date" @bind="condition.Date" readonly="@(!isEditMode)" /></td>
                        <td>
                            <select @bind="condition.Status" disabled="@(!isEditMode)">
                                @foreach (var status in Enum.GetValues(typeof(Status)))
                                {
                                    <option value="@status">@status</option>
                                }
                            </select>
                        </td>
                    </tr>
                }
            }
            </tbody>
        </table>

        <button class="btn btn-primary btn-sm" @onclick="ToggleEditMode">@editButtonText</button>
        @if (isEditMode)
        {
            <button class="btn btn-success btn-sm" @onclick="SaveChanges">Save</button>
        }
        </div>
}

@code {
  [Parameter]
  public string Id { get; set; }

  private Patient? patient;
  private bool isEditMode = false;
  private string editButtonText = "Edit";

  protected override async Task OnParametersSetAsync()
  {
    if (!string.IsNullOrEmpty(Id))
    {
      patient = await MongoDbService.GetPatientAsync(Id);
    }
  }

  private void ToggleEditMode()
  {
    isEditMode = !isEditMode;
    editButtonText = isEditMode ? "Cancel" : "Edit";
  }

 private async Task SaveChanges()
 {
   if (patient != null)
   {
     await MongoDbService.UpdatePatientAsync(patient.Id.ToString(), patient);
    isEditMode = false;
    editButtonText = "Edit";
   }
 }
}

A lot of this code is still the same, but it now has a boolean for isEditable which toggles edit mode on and off, as well as buttons to edit, save, or cancel the changes and calls our UpdatePatientAsync method in the service class.

Delete a patient

Finally, it is time to add the ability to delete a patient. First, we will add the method inside MongoDBService.cs:

public async Task DeletePatientAsync(string id)
{
  if(_patientsCollection == null)
    throw new InvalidOperationException("Patients collection is not initialized");

  await _patientsCollection.DeleteOneAsync(p => p.Id == ObjectId.Parse(id));
}

Then, we are going to add a delete button in two places: in the list of patients in Patients.razor and on the individual patient in PatientDetails.razor.

Let’s add a delete button to our Patients.razor page first. Within the table block, update the foreach loop to add the following delete button after the existing view details button:

<button class="btn btn-danger btn-sm" @onclick="() => DeletePatient(patient.Id)">Delete</button>

This will error for now as we next need to implement the DeletePatient method in the code block:

private async Task DeletePatient(ObjectId patientId)
{
  await MongoDbService.DeletePatientAsync(patientId.ToString());
        patients = await MongoDbService.GetPatientsAsync();
}

Next, we will add the button to the PatientDetails.razor page. Add the following code to the page, after the if check for isEditMode:

<button class="btn btn-danger btn-sm" @onclick="() => DeletePatient(patient.Id)">Delete</button>

Then, add the following method to the code block on the page, to call the service class and navigate back to the list of patients on deletion:

private async Task DeletePatient(ObjectId patientId)
{
  await MongoDbService.DeletePatientAsync(patientId.ToString());
        NavigationManager.NavigateTo("/patients");
}

Testing

Now we have everything in place, it is time to test it out. Run the application and give it a try.
The home page has links to view patients and add patients that you can visit.

If you add a patient and then view your cluster, there will be a database called medicalRecords and a collection in that database called patients where you will find your newly inserted document, complete with encrypted fields! This is queryable encryption at work. The data is encrypted in the database and in transit, but because the application has access to the keys used to encrypt the data, it isn’t encrypted when displayed!

You can even edit a patient document from the PatientDetails page now, or delete any existing patients, and you will see all the changes reflected in your collection.

Summary

Just like that, we have a working application showing queryable encryption in action. The QueryableEncryptionHelpers.cs class can even be reused across applications so you have a head start to make you super productive!

Then, all you need to do is configure the encryption settings and encrypted fields and continue to use it as you would with the C# driver without any encryption. So the barrier to entry for queryable encryption is really low.

The full code can be found in the GitHub repo on the with-queryable-encryption branch.

Why not start applying it to your enterprise applications today? If you have any questions or want to share how you got on, you can visit our Community Forums.

DEV Community