Announcement: Razor Compiler API Breaking Changes #8399
Description
Over the past year, the Razor team has been on a journey to improve performance and reliability of the compiler, paying down technical debt, reorganizing infrastructure to enable performance tracking and faster shipping, and restructuring compiler internals to reduce complexity and improve compiler performance. While we've had a good amount of success in this area, architectural decisions made for the current version of the compiler and maintaining compat with these decisions are limiting our ability to make the even bigger improvements we need to make to deliver a good Razor experience. Therefore, we will be freezing our public APIs and the packages we publish for the compiler, and we will publish new packages in the future with public APIs we can support going forward.
High Level Changes Overview
Before we get into the details of why we need to make this change and the specifics of the timeline, we’d like to give a high level overview of the change:
- We will be working on a new public API more closely resembling the Roslyn model, with goals to improve performance and enable new functionality such as analyzers and refactorings in Razor files. When we are confident in our public API for these scenarios, we will start publishing new Razor compiler packages.
- Existing Razor compiler packages will not have new versions published. Existing versions will continue to exist and be serviced according to the standard .NET servicing guidelines. You can continue consuming these packages for your existing scenarios without any changes.
- As part of this there are no changes to how, and where, we build the Razor compiler. The code remains open source, and we welcome community contributions in the form of issues, PRs, etc.
The rest of this issue will cover the specifics of why we’re doing this, the timeline of the change, and the exact packages this will affect.
Incompatible Design Goals
The modern Razor compiler was designed to be extremely extensible: users can substitute their own versions of many components of the compiler pipeline, using the Razor HTML templating syntax with their own embedded language and emitting to their own languages. While this was an important part of the initial vision of Razor, it does mean that the public API components of the compiler are extensive, and rearchitecting how compilation phases interact with each other, the information each phase is responsible for building, and the format of the output of each phase is potentially a breaking change. Some examples of where the existing structure impacts our experiences are:
- Semantic information about the project the Razor file is embedded in is exposed in the syntax tree model itself. Syntax types such as
MarkupTagHelperAttributeSyntaxdepend on knowing what tag helper types exist; this means that, with the current architecture, Razor files cannot be parsed without crawling the entire containing project for tag helper types. This then impacts our ability to give even basic syntax highlighting before a significant amount of expensive work on the containing project has been done. - Another parser-related architecture component, error recovery becomes incredibly complex (even more so than it already is). Because any language can be embedded within Razor's HTML markup, it limits our ability to solve issues like auto-completing div tags doesn't work properly in nested scenarios #7608, a top issue for Razor users. In order to handle error recovery for these cases, the HTML and C# parsers will need to work in tandem and be more tightly coupled than they are today.
- The existing compiler phases have a number of structures embedded in the public API that have proven inefficient. Some of these could just be deprecated and eventually removed, but some of them, such as the structure of the tokenizer and passing of semantic information, have core issues and will need to be reworked to be usable.
Given these and other similar issues, we no longer believe that the original goal of allowing any embedded language to be used with the Razor HTML components is something that we can support going forward. Blazor support already started down this path, as it deeply integrates the Razor syntax with C#, and we think this provides a good experience for users. We will be focusing on making the Razor + C# experience as awesome as possible and addressing the above issues will be key to this effort.
New Design Goals
Despite this change, the Razor team does believe in the power of public compiler APIs and tooling based on them. Many of us that work on the Razor team also work on the C# compiler and tooling teams and have first-hand experience with awesome features that can be enabled by good public compiler APIs (indeed, many of us have been part of the design of the APIs that analyzers and source generators use today). We would like to enable a similar level of API for interacting with Razor files: being able to interact with a Razor syntax tree, query semantic information about the tree, and getting information about embedded C# as well. We hope to enable writing analyzers and fixers for Razor files, much like people can write analyzers and fixers for C# files with Roslyn today. It will take some time for us to get to this point; restructuring a hugely popular compiler without breaking users isn't an overnight task. But the new structure will be built with this goal in mind.
Timelines and Actions
Currently, the Razor compiler and supporting packages are published to Nuget.org, in addition to being included with the .NET SDK and VS. This will change:
- VS 2022 Version 17.6 - We will not publish new versions of the existing Razor compiler and source generator packages on Nuget.org. They will still be present in VS and the SDK in the same shape as they are currently to support old editing experiences, but they will not receive new changes. You can continue using these packages for your existing scenarios, but they will not be receiving new Razor features. These packages are:
- VS 2022 Version 17.7 - We will restructure the dlls that the compiler produces into new names, likely
Microsoft.CodeAnalysis.Razor.Compiler, consolidating our different dlls. This new package will start receiving breaking changes as soon as our partner IDE and runtime compilation experiences have absorbed the changes. This package will not be published to Nuget.org yet, as we make major breaking changes to its internals. - VS 2022 Future - When we have restructured the compiler, worked out the kinks, and designed/vetted our public API structure, we will start publishing the new Razor compiler package to Nuget.org.
Of course, these packages will be available on our dev feeds, for adventurous users, but they will come with breaking changes, possibly even breaking new APIs as we continue restructuring.
For those who made it all the way to the end, thanks for reading. These changes, while big, will help us bring a better experience for Razor consumers, and a more usable public API.