Guidance on how to log to a message queue for slow data stores

[Fniz]

Can you give me more precision regarding this sentence : "If your logging data store is slow, don't write to it directly. Consider writing the log messages to a fast store initially, then move them to the slow store later. For example, log to a message queue that's read and persisted to slow storage by another process."

I want to store log in SQL server, so I suppose it's a slow data store. So I would like what you mean by "log to a message queue". Do you reffer to RabbitMQ for example ?

Yours sincerely,

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 7184a308-a254-9350-a5cd-5bced1f369ae
• Version Independent ID: 726e3bf1-f367-d733-8933-bccc04da0e16
• Content: Logging in ASP.NET Core
• Content Source: aspnetcore/fundamentals/logging/index.md
• Product: aspnet-core
• Technology: aspnetcore-fundamentals
• GitHub Login: @tdykstra
• Microsoft Alias: tdykstra

[Rick-Anderson]

@tdykstra do you want to take this issue and update the article to say
Log to a message queue (such as ...)

[tdykstra]

@Rick-Anderson Yes, good idea.

[Fniz]

Thank you very much for your answer. Is there non-cloud credible alternative to Azure service bus ?

[analogrelay]

Actually this refers to using in-memory queuing. If you are logging to SQL server, you don't want to do that directly in the Log method, since it's synchronous. Instead, you should (synchronously) add the log message to an in-memory queue and have a background worker come along to pull the items out of the queue and do the asynchronous work (pushing data to the SQL server).

[Fniz]

Thank you @anurse for your answer. When you talk about an in-memory queue, is it like this for the ConsoleLogger : https://github.com/aspnet/Logging/blob/master/src/Microsoft.Extensions.Logging.Console/Internal/ConsoleLoggerProcessor.cs

[analogrelay]

@Fniz precisely right! Even the console logger ended up finding improvements by using an in-memory queue. A SQL (or other remote) backend would definitely want something similar.