ASP.NET template disables EF Core SQL logging by default

[Rick-Anderson]

Include your code

Create an ASP.NET Core app with individual user accounts so EF Core is used

dotnet new webapp -au Individual -uld -o RPauth
cd RPauth
dotnet ef database update
dotnet run

Browse to https://localhost:5001/ and create a new account. Click on the link to confirm the account. Note that no SQL is logged.

Add the following to appsettings.json:
,"Microsoft.EntityFrameworkCore.Database.Command": "Information"

No need to stop/restart as appsettings.json is reloaded. Navigate to https://localhost:5001/Identity/Account/Manage can select some of the tabs on the left. SQL output is displayed.

Comment out the following and SQL output stops:

// ,"Microsoft.EntityFrameworkCore.Database.Command": "Information"

Include provider and version information

Using all defaults with .NET 5 or .NET 6

Related: dotnet/efcore#3240

[ajcvickers]

Moving this to ASP.NET since the configuration generated explicitly circumvents the default SQL logging from EF Core by setting the default level for all Microsoft events to Error. We had agreed when this was implemented that EF Core would log SQL at Info, and ASP.NET would show this by default.

[javiercn]

Thanks for the additional info.

I've tagged this as MVC, although its for the team to handle.

@ajcvickers what namespace are the queries logged in from. I imagine we need to add an additional entry to the appsettings.json to mark those logs as info and have the queries show up.

[Rick-Anderson]

I imagine we need to add an additional entry to the appsettings.json to mark those logs as info and have the queries show up.

See SQL Logging of Entity Framework Core
,"Microsoft.EntityFrameworkCore.Database.Command": "Information"

[mkArtakMSFT]

@davidfowl , @DamianEdwards looks like you had changed this area in the past so what do you want to do regarding this?

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[davidfowl]

Pretty sure this is by design. We don't want logs showing up as information by default. It's too much noise.

[ajcvickers]

It's by-design that EF Core shows SQL by default. Not doing so leads to things like the stackoverflow question that triggered this. We agreed with ASP.NET that SQL would be shown by default if we logged at Info level.

[Rick-Anderson]

Pretty sure this is by design. We don't want logs showing up as information by default. It's too much noise.

So add // ,"Microsoft.EntityFrameworkCore.Database.Command": "Information"
to appSettings.development.json

See this SO thread, 50K + views, all the ridiculous answers to get SQL logging. Luckily I was able to contact the owner and have him change my answer to the correct answer.

See SQL Logging of Entity Framework Core - Probably documenting it is enough. That just went live today and it's an include file so it's all over the ASP.NET Core EF tutorials.

[Rick-Anderson]

I have extensive anecdotes that customers want to see SQL logging in development, that's fundamentally different than seeing framework logging. The best fix might be enabling it in appSettings.Development.json and then documenting how to turn it off.

What would be really cool and beneficial is adding several useful logging flags // commented out in appSettings.Development.json with a link to a troubleshooting document describing them. I've seen lots of issues opened where the response is, "Enable this flag to debug".

What customers do now, after alot of searching, is enable logging of all of Microsoft*, then start backing off.

We really need a trouble shooting section under Configure logging with details on the important config settings and suggestions.

[davidfowl]

We don't show anything info level by default, it doesn't have anything to do with EF, we literally don't show per request logs. Maybe this is something we can change in development but not in production? Logging per request is extremely inefficient and we opt for showing the user's logs by default but not the framework logs.

[Rick-Anderson]

Maybe this is something we can change in development but not in production?

My original suggestion, add ,"Microsoft.EntityFrameworkCore.Database.Command": "Information"
to appSettings.Development.json

[ajcvickers]

My preference would be to put all of EF Core at Info, at least for development. That is, "Microsoft.EntityFrameworkCore"

[davidfowl]

I'm not sure we want that much noise. This is something that we should discuss 😄

[DamianEdwards]

In 2.x, the templates are set so the logging level for Microsoft prefixed categories is Information in appsettings.Development.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Debug",
      "System": "Information",
      "Microsoft": "Information"
    }
  }
}

In 3.x, this was changed to reduce the amount of logging emitted by default, to be Warning, with a specific setting added to ensure Microsoft.Hosting.Lifetime messages would still be emitted (so it was easy to see that the app had started successfully in the console during development):

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  }
}

This would have resulted in EF logs that were previously emitted by 2.x apps to not be emitted in new 3.x apps (and beyond).

Personally, I agree it's very useful to have some EF logs present in development by default and support adding a specific setting to the templates to emit EF Information logs. My initial thought is I'd only expect a single log per query executed, which would include the SQL command text. Thus we'd need to look at what EF emits at the Information level today ensure the settings file entry is specific enough to result in the desired amount of logging (and potentially update EF if that's not possible right now).

[ajcvickers]

EF Core logs the following at Information level:

• All providers:
  • Entity Framework Core {version} initialized '{contextType}' using provider '{provider}' with options: {options}
  • A transient exception occurred during execution. The operation will be retried after {delay}ms.{newline}{error}
• Cosmos:
  • Reading resource '{resourceId}' item from container '{containerId}' in partition '{partitionKey}'.
  • Executing SQL query for container '{containerId}' in partition '{partitionKey}' [Parameters=[{parameters}]]{newLine}{commandText}
• Relational:
  • Executed DbCommand ({elapsed}ms) [Parameters=[{parameters}], CommandType='{commandType}', CommandTimeout='{commandTimeout}']{newLine}{commandText}
  • Applying migration '{migration}'. (Migrations only)
  • No migrations were applied. The database is already up to date. (Migrations only)
  • Reverting migration '{migration}'. (Migrations only)
  • The index named '{indexName}' on the entity type '{entityType}' specifies properties {indexProperties}, but none of these properties are mapped to a column in any table. This index will not be created in the database. (Will follow up on why this is at Info level. /cc @AndriySvyryd) Fixed.
  • The unnamed index on the entity type '{entityType}' specifies properties {indexProperties}, but none of these properties are mapped to a column in any table. This index will not be created in the database. (Likewise) Fixed.

[DamianEdwards]

Thanks @ajcvickers. From that list it seems that for a typical web app then (e.g. the Identity pages) there'd be 2 log messages emitted per request, one for the DbContext initialization and another for the query execution. Are they logged form different categories? If so, how do folks feel about just adding the setting to enable the relational query execution log?

[ajcvickers]

@DamianEdwards Yes, I think that is correct, and they are in different categories. The generated SQL for both Cosmos and relational is in the category, Microsoft.EntityFrameworkCore.Database.Command, so enabling this will get just the SQL back. I'm fine with this (it is was what @Rick-Anderson suggested above), although I slightly prefer turning all Info events.

[DamianEdwards]

Cool so this is a fairly simple template change then. Does anybody have any concerns with a logging setting for EF Core being present in the templates even when EF Core itself isn't referenced? I don't in particular but wanted to ask anyway.

[javiercn]

Not really, it's also not hard either to just enable it when EF is involved in the template. Its just a switch for IndividualAuth

[Rick-Anderson]

Not really, it's also not hard either to just enable it when EF is involved in the template. Its just a switch for IndividualAuth

We don't always know when EF is being used. Get started with Razor Pages in ASP.NET Core doesn't use auth. Customers following that tutorial want EF logging enabled so I can remove SQL Logging of Entity Framework Core from the tutorial.

[ajcvickers]

Agree with @Rick-Anderson here. Numbers show that most people using EF Core with ASP.NET Core are doing so without using Identity. We should not limit this to only when Identity is included in the template.

[davidfowl]

I think we need to see what typical per request logs look like for a simple crud application. That'll help determine if we should reduce the noise or not.

[ajcvickers]

@davidfowl It will look that same as it does now with one extra log message for any SQL query.

[Rick-Anderson]

What's the harm of SQL noise in development? I can splatter the docs with instructions on how to comment out EF logging.

[DamianEdwards]

@Rick-Anderson not sure anyone is suggesting there's harm, just trying to ensure all folks find the defaults reasonable from their, admittedly subjective, view point.

[davidfowl]

There may harm. People do things like run performance tests in development. Also if there's too much noise then logs trend towards being not as useful

[DamianEdwards]

@davidfowl indeed, but let's scope the conversation to what we know the delta is that's being discussed here. Are you pushing back or are you OK with it?

[davidfowl]

Yes, those logs look fine.

[Rick-Anderson]

Now that we have #32977 :)

[ghost]

Thanks for contacting us.

We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[gurkantuna]

Hey @Rick-Anderson, I found your answer on SO while researching about ToTraceString(). I was going to write there but you know, the comments are limited within text there. Although I can monitor the query from Server Profiler as below, I cannot see it from the output window.

I added the "Microsoft.EntityFrameworkCore.Database.Command": "Information" line to the configuration file, but I still cannot see the generated query. By the way, you have shown that you have selected the project in the output window.

I'm probably missing the point because of I am debugging with test project in debug mode and there is no project names.

Since I used Server Profiler all the time before, I never needed it, but I wanted to use it after I saw your answer. So, what do you think I'm missing?

[DamianEdwards]

@g-u-r-k-a-n the capturing of app logs and showing in the Output window only works when launched without debugging. You can however seem them in the Output window during debugging in the "Debug" source. Once selected right-mouse click on the content to turn off the messages that aren't relevant (e.g. module loading).

[gurkantuna]

Thanks @DamianEdwards . I knew them. Even the exceptions are selected. All of them are like that. I'm in the Debug mod. You describe exactly what i'm doing right now ☺️ In fact, there are no any message except the modules. I followed your advice and turned others off but situatiuon of output window is the same.

[DamianEdwards]

@g-u-r-k-a-n can you share the contents of your appsettings.Development.json file please?

[gurkantuna]

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information",
      "Microsoft.EntityFrameworkCore.Database.Command": "Information"
    }
  },
  "AllowedHosts": "*",
  "ConnectionStrings": {
    "DefaultConnection": "Server=.;Database=BlogDB;User Id=sa;Password=123;"
    //"DefaultConnection": "server=.;Database=BlogDB;Trusted_Connection=true;MultipleActiveResultSets=True;"
    //,"DefaultConnection": "server=(localdb)\\mssqllocaldb;Database=BlogDB;Trusted_Connection=true;MultipleActiveResultSets=True;"
  }
}

@DamianEdwards I was using different settings from config file but I turned to default config after your advise. So, I've just added others to try different ConnectionStrings but I closed them with comments as is seen. By the way, I have another base DbContext in the other .NET 5 project. I'm talking about a project that provides my basic needs. I'm building appsetting.json in OnConfigure() from that project. I debugging it and the connections settings is fetching succesfully.

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) {
    var configuration = new ConfigurationBuilder()
                            .SetBasePath(AppDomain.CurrentDomain.BaseDirectory)
                            .AddJsonFile("appsettings.json")
                            .Build();

    optionsBuilder.UseSqlServer(configuration.GetConnectionString("DefaultConnection"));
}

[DamianEdwards]

Is there a particular reason you're overriding OnConfiguring to do the DbContext configuration rather than using the built-in methods on IServiceCollection to configure this from your Startup class? Do you see any log messages at all for the ASP.NET Core application in the output window (either for Debug or the project itself)?

[gurkantuna]

@DamianEdwards This was what I tried to explanation it with the 'Base' idiom in the previous comment. in fact, you can think actual name of that project is Company.Core. Because I have a my own Core Project. I don't mentioned .NET Core, I've been using the 'Core' name for a long time already. I prefer this way to solve my similar requirements or concerns from single point. I'm updating it when I need a new similar requirement. I have some settings or infrastructure concernes in that project and I don't want to break this struct . In this way, I can concentrate to the essential creating the algorithms independently from the technologies.

[DamianEdwards]

@g-u-r-k-a-n sorry, I'm a bit confused now. Are you still having an issue getting the EF Core logs to show up? If so, can you share the details of the project that is executing including it's appsettings.json and appsettings.Developer.json logging sections, and the code in your Program.cs and Startup.cs that configures the host/logging and the services collection?

[chengyanbing]

When I use a real database such as "sql server", it works very well, but when I use MemoryDatabase like this:
services.AddDbContext(options => options.UseInMemoryDatabase("Customers")),
it doesn't work. Is that how it was designed? Or What did I miss?

[ajcvickers]

@chengyanbing The in-memory database is not a relational database and does not use SQL, so there is no SQL to log.