JSON, JavaScript.

Azure, EF Core 8, SQL Server, REST, SOAP

MAY
JUN
2024

The DNA . YEARS

of a Database
Developer

Using Complex Working Understanding

Properties w ith JSON Azure
in EF Core 8 Documents Migrations
CODE
CONSULTING

ARE YOU WONDERING

HOW ARTIFICIAL
INTELLIGENCE CAN
BENEFIT YOU TODAY?
EXECUTIVE BRIEFINGS
Are you wondering how Al can help your business? Do you worry about privacy or regulatory issues stopping
you from using Al to its fullest? We have the answers! Our Executive Briefings provide guidance and
concrete advise that help decision makers move forward in this rapidly changing Age of Artificial Intelligence
and Copilots!

We will send an expert to your office to meet with you. You will receive:
1. An overview presentation of the current state of Artificial Intelligence.
2. How to use Al in your business while ensuring privacy of your and your clients' information.
3. A sample application built on your own HR documents - allowing your employees to query
those documents in English and cutting down the number of questions that you
and your HR group have to answer.
4. A roadmap for future use of Al catered to what you do.

AI-SEARCHABLE KNOWLEDGEBASE AND DOCUMENTS

A great first step into the world of Generative Artificial Intelligence, Large Language Models (LLMs),
and GPT is to create an Al that provides your staff or clients access to your institutional knowledge,
documentation, and data through an Al-searchable knowledgebase. We can help you implement a first
system in a matter of days in a fashion that is secure and individualized to each user. Your data remains
yours! Answers provided by the Al are grounded in your own information and is thus correct and applicable.

COPILOTS FOR YOUR OWN APPS

Applications without Copilots are now legacy!
But fear not! We can help you build Copilot features into your applications in a secure and integrated
fashion.

CONTACT US TODAY FOR A FREE CONSULTATION AND DETAILS ABOUT OUR SERVICES.

codemag.com/ai-services
832-717-4445 ext. 9 • [email protected]
 TABLE OF CONTENTS

Features
7 CODE: 20 Years Ago 46 Stages o f Data: The DNA o f
Markus c o n tin u e s his re fle c tio n on w h a t th e com pany, th e m agazine,
and th e in d u s try have been up to fo r th e la s t th re e decades.
a Database Developer, Part 1
Markus Egger W hether yo u 're g o in g to an in te rv ie w as th e a p p lic a n t o r th e
in te rv ie w e r, y o u 'll be glad t h a t Kevin came up w ith th is c o lle c tio n
o f th e th in g s y o u o u g h t to know i f y o u w a n t to succeed.
10 Async Programming in JavaScript Kevin Goff
S a h il shows yo u how to c o o rd in a te th e m u ltip le processors th a t
yo u need to do a n y th in g in to d a y's h ig h -p a ce d c o m p u tin g w o rld .
Sahil Malik
59 From SOAP to REST to GraphQL
I f y o u need to store, m ove, o r access d a ta , y o u 'll need to know
how to make sure t h a t a ll o f y o u r system s ta lk to each other.
16 M anipulating JSON Documents J o y d ip e xplains how SOAP, REST, and GraphQL co m b in e to make
t h a t a sm ooth process.
in .NET 8 Joydip Kanjilal
Ja va S crip t O b je ct N o ta tio n (JSON) can help y o u c o n fig u re s e ttin g s
and tra n s fe r d a ta , b u t i t re a lly shines w hen i t comes to c re a tin g
and m a n ip u la tin g docum ents in .NET 8. Paul shows y o u how.
Paul Sheriff

33 Value Object's New Mapping:

Departments
EF Core 8 ComplexProperty Editorial
J u lie looks a t th e m any changes in EF Core 8 t h a t were released a t
th e end o f Last ye a r and fin d s t h a t C om plexP roperty m apping re a lly
takes th e cake. Advertisers Index
Julie Lerman

Code Compilers
38 Preparing for Azure w ith
Azure Migrate Application
and Code Assessment
Your com pany is s w itc h in g to p la tfo rm -a s -a -s e rv ic e (PaaS) from
on-prem ises and y o u need to re -p la tfo rm y o u r a p p lic a tio n s .
Mike shows yo u how to make t h a t happen using Azure M igrate
A p p lic a tio n and code assessm ent.
Mike Rousos

US subscriptions are US $29.99 fo r one year. Subscriptions outside the US pay

$50.99 USD. Payments should be made in US dollars drawn on a US bank. American
Express, MasterCard, Visa, and Discover credit cards are accepted. Back issues are
available. For subscription in form ation, send e-m ail to [email protected]
or contact Customer Service a t 832-717-4445 ext. 9.

Subscribe online a t www.codemag.com

CODE Component Developer Magazine (ISSN # 1547-5166) is published bim onthly

by EPS Software Corporation, 6605 Cypresswood Drive, Suite 425, Spring, TX
77379 U.S.A. POSTMASTER: Send address changes to CODE Component Developer
Magazine, 6605 Cypresswood Drive, Suite 425, Spring, TX 77379 U.S.A.

4 Table of Contents codemag.com

 W HY C H O O S E LE A D TO O LS

Introducing LEADTOOLS Excel

API and W eb Editor
LEADTOOLS V23 now lets you integrate any Excel workflow into your own custom
application and programmatically load, create, edit, and save spreadsheets.

Web-native Excel Experience

Seamlessly transition from desktop spreadsheet to
browser-based with the same familiar functionality and an
intuitive interface.

Full Excel Compatibility

Whether you're working with XLS or XLSX, LEADTOOLS provides
smooth read/write operations and robust Excel functionality.

Rich Excel Operations

=NOW ()+5
Create and edit sheets, update styles, manage formulas,
manipulate im.ages and more as LEADTOOLS comes complete
with a comprehensive set of tools.

O ther LEADTOOLS SDK features include

Get S ta rte d Today

LEADTOOLS.COM
DOWNLOAD OUR FREE EVALUATION
 EDITORIAL

Unfinished Paintings
I spent last weekend in rainy (normally sunny) Southern California. During this trip, I managed to corral
the kids into going to the Academy Awards Museum. There, we came across a set of drawings from the
original Disney animated film The Little Mermaid (Figure 1). From what I could deduce from the drawings,

I believe they are what are called key frames. tarps like you can buy a t W almart), I saw street So being the fresh-faced coder, how could
In anim ation, the artists draw key frames, which posters, statues, and oth er paintings from his I learn to b e tte r my craft? I did th is several
represent different transitions in the animation extensive set o f works. One o f these works ways. I read every book I could get my hands
and are then passed onto other artists who f ill in stood o u t to me. Figure 2 is an image o f an on, I w ent to the "b ig " c ity o f Portland where
the frames between the keys. As I described what unfinished work started n o t soon before his un­ I discovered knowledge heaven in the form o f
these frames were for to my kids (and a random tim e ly death. I spent tim e studying the image, Powell books, and, fina lly, I read every com­
onlooker), I discovered mentally why I'm fasci­ looking a t i t from d iffe re n t angles and vantage puter magazine I could get my hands on. The
nated w ith such works. I'm fascinated by them points. I trie d to picture Keith w orking on th is articles th a t I really to o k a Liking to were the
because they're unfinished. I find th a t these a rti­ p a in tin g in his studio in the flo w o f a rtis tic cre­ ones where the authors explained the process
facts o f the creative process give me insights in to a tiv ity . This image to o k me in to a creative flo w o f how they achieved a solutio n. Using a rt as
the mind o f the a rtist creating them. state. A ll from an unfinished pa in ting . a metaphor, they to o k me through the rough
drawings, pencil sketches, rough demos, base
I Love the rough drawings, erasure marks, rough For some reason, I've always loved works from coats, d e ta il work, and fin a lly, a finished
lines, and a ll the th in g s th a t show how the a rt­ a rtists th a t are incom plete works versus com­ w orking solutio n. I was w atching techno a rt­
is t works. This to o k me back to another mu­ pleted works. This is my in n e r creator coming ists take me through th e ir process from u n fin ­
seum v is it. out. I t takes me back to the earliest parts o f ished to finished work. This is my process to
my career where I was the "Lone w o lf' devel- th is day.
Last year I was lucky enough to attend an exhi­ op er/n etw ork admin fo r a sm all resort in Cen­
b itio n o f the works o f Keith Haring a t the Broad tra l Oregon. This was back in the Late 80s when Many o f the unique solutions I've b u ilt over
Museum in downtown Los Angeles. I've been a there was no in te rn e t Like we have now. There the years come from a process like th is one.
follow er o f Keith's work fo r decades now and were no GitHub repositories, no code blogs, I'm tasked w ith seeing i f an idea m ig h t work.
d o n 't miss any chance to see collected exhibits no www.stackoverflow.com . Nope. There were For instance, many years ago, I was tasked
o f his work. This e x h ib it stood o u t to me as crappy 1200 or 2400 baud modems by which we w ith bu ildin g a solution where we embedded
I saw works and materials he'd used th a t I'd reached o u t in to the world to gather our knowl­ code in M icrosoft Word documents th a t gave
never read about or witnessed. I saw fu ll blown edge from forums like CompuServe and Genie. I t users the a b ility to create dynamic scripts fo r
murals painted on camping tarps (yes ,camping was the dark ages. LOL. call centers. I started th is process to see i f I
could embed code in to a work document. This
was my rough sketch. I then too k the o u tp u t
from th a t docum ent and b u ilt an HTML-based
scrip t using the metadata embedded in the
docum ent, another sketch. I then combined
these tw o to g e th e r in to a rough demonstra­
tio n fo r the clie n t. The c lie n t liked w hat they
saw, and we w ent through the process repeat­
edly u n til we had a good w orking so lu tio n . This
code lasted nearly 10 years u n til a new solu­
tio n was im plem ented. I t enjoyed many years
o f success, a ll o rig in a tin g from a rough sketch
o f code.

This process has worked fo r me tim e and tim e

again. I love ju s t try in g s tu ff to see i f i t works.
I f i t does, I file i t away fo r future use; i f not, I
Look a t other solutions. I'm always experim ent­
in g, always sketching, always creating u n fin ­
ished paintings.

Figure 1: A key frame Figure 2: An unfinished work by Keith Haring

6 Editorial codemag.com
 ONLINE QUICK ID 2405011

CODE: 20 Years Ago

The end of 2023 was the start of our "30 years of CODE" celebration year, which will continue throughout all of 2024. To look
back at those 30 years, I wrote articles in the last two issues of CODE Magazine, looking at what happened 25 and 30 years ago.
This time, I'll look back 20 years and explore the latest and greatest of the early-to-mid 2000s. I remember it as a somewhat

tu rb u le n t tim e. The dotcom bubble had burst, and the projects we were w orking on in the consulting and custom
glory days o f technology seemed to be over. Was the in ­ app dev side o f the business w eren't classic dotcom com­
te rn e t really a ll i t was supposed to be or was i t ju s t a panies. Also, we'd started CODE Magazine in the Spring o f
passing fad? I t was hard to say. 2000 and focused prim arily on the new world o f software
developm ent th a t M icrosoft was generating. The Java pro­
gramming language was o f in te re st to a Lot o f people b u t
The Aftermath of the Dotcom Bubble had some issues th a t were, as o f the n, unaddressed, and
One o f the main catalysts fo r the dot-com -bubble bursting one way to fix i t was Microsoft's approach o f re-in ven t­
was the overvaluation o f many internet-based companies ing the language in a top-secret project headed up by
th a t had little or no profits b u t huge expectations. Inves­ language-guru Anders Hejlsberg, codenamed "C ool." (This Markus Egger
tors poured money in to these ventures, hoping to cash in became C#, and yes, C# is s till cool. You may have seen [email protected]
on the next big th in g , b u t many o f them turned o u t to be the T -shirt).
unsustainable or unprofitable. Some o f the most notorious Markus, the dynamic
examples o f dotcom failures were Pets.com, Webvan, eToys, C# became a key com ponent o f the then nascent .NET founder of CODE Group and
CODE Magazine's publisher,
and Boo.com, which burned through m illions o f dollars in ecosystem, which did away w ith the concept o f the pro­
is a celebrated Microsoft
a m atter o f months before going bankrupt. The collapse o f gramming Language driving everything and instead cre­
RD (Regional Director) and
these and other companies sent shockwaves through the ated a developm ent framework th a t could be used equally
multi-award-winning MVP
stock market, w iping ou t billions o f dollars in value and from various languages. This was a concept th a t jiv e d
(Most Valuable Profession­
causing many investors to Lose confidence in the sector. very much w ith w hat we believed a modern software de­
al). A prolific coder, he's
velopm ent magazine should be ta lk in g about, and thus
influenced and advised top
The world o f software development wasn't immune to the CODE Magazine found its e lf in a sweet spot o f sorts. Other Fortune 500 companies
effects o f the dotcom bubble bursting. Many software magazines, Like Visual Basic Programmer's Journal, Fox­ and has been a Microsoft
developers who'd been hired by dotcom startups found Pro Advisor, and many more, suddenly d id n 't look so hot contractor, including on
themselves o u t o f work when th e ir employers w ent under. anymore. A Lot o f th is w asn't a coincidence. A fte r all, the Visual Studio team.
Some o f them had to accept Lower salaries or switch ca­ we'd long been partnering very closely w ith M icrosoft— I Globally recognized as
reers, and others tried to start th e ir own businesses or jo in worked fo r the Visual Studio team as a contractor on va ri­ a speaker and author,
more established companies. The demand fo r web develop­ ous projects— and we were strong believers in these new Markus's expertise spans
ment skills decreased, as many companies scaled back or concepts. A rtificial Intelligence (A l),
canceled th e ir online projects. The failure o f many do t­ .NET, client-side web, and
coms also raised questions about the v ia b ility and quality A ll these goings-on meant th a t we were somewhat pro­ cloud development, focus­
o f some o f the emerging web technologies and standards, tected from the dotcom mess. Yes, we also lo st some cus­ ing on user interfaces,
such as Java, XML, and HTML. Some critics argued th a t tomers, and the pool o f p o te n tia l new customers shrank. productivity, and maintain­
these technologies were overhyped and underdelivered, We had to tig h te n our belts a b it, b u t overall, we came able systems. Away from
and th a t they weren't suitable fo r building complex and through i t a ll reasonably w ell. I remember i t as a tim e work, he's an enthusiastic
windsurfer, scuba diver,
reliable applications. Others defended these technologies th a t was p a in fu l fo r us, b u t n o t to an e xiste n tia l level.
ice hockey player, golfer,
and claimed th a t they were s till evolving and im proving, And despite a ll the in te rn e t disillusionm ent, we remained
and globetrotter, who
and th a t they would eventually prove th e ir w orth. s to u t believers th a t i t w asn't the in te rn e t th a t was the
Loves gaming on PC or
problem b u t rather the problem was the idea th a t the
Xbox during rainy days.
Despite the challenges and setbacks th a t the dotcom bub­ in te rn e t made economic fundam entals obsolete. In other
ble bursting posed for the software industry, i t also had words, we considered i t crucially im p o rta n t to push fo r­
some positive effects. I t forced many companies to rethink ward w ith in te rn e t-re la te d technologies. As a M icrosoft-
th e ir business models and strategies, and to focus more on focused organization (and a M icrosoft partner), th is
customer needs and satisfaction, rather than on growth meant m ainly focusing on ASP.NET as the backbone o f
and hype. I t encouraged more innovation and experimenta­ alm ost a ll web applications th a t we wrote. We had largely
tio n as some developers sought to create new and better ignored earlier versions o f ASP, b u t then there was th is
solutions for the web. I t also paved the way fo r the emer­ young kid o f a program manager s tra ig h t o u t o f college
gence o f new players and platforms, such as Google, Ama­ w ith a vision o f a b e tte r web developm ent environm ent. I
zon, eBay, and PayPal, which took advantage o f the oppor­ was very impressed w ith his early demos. He was a funny
tu n itie s and gaps in the market th a t the dotcom crash had and rather Likable kid, and he always wore red shirts. His
le ft behind. These companies would go on to become some name was S cott someone or another. I th in k he s till works
o f the most successful and in flu e n tia l in the history o f the a t M icrosoft today. <grin>
in te rn e t and to shape the future o f software development.
And before you ask, most o f the web applications we
wrote in those days were m ainly b u ilt fo r In te rn e t Explor­
The Impact for CODE er, the de fa c to standard browser o f the tim e. Netscape
Luckily fo r us a t the CODE Group, the dotcom turbulences had faded in im portance as they Lost the "g rea t browser
were less severe than fo r oth er companies. Most o f the wars" against M icrosoft, and Firefox w asn't a th in g yet.

codemag.com CODE: 20 Years Ago 7

 Internet Explorer had taken the web and HTML from being market, but also a steady horse to bet on for a while. I
a simple mechanism to displaying hyperlinked text and remember consulting around quite a few Microsoft tech­
simple documents th a t supported only laughable Levels nologies back then, and they were all solid investments. I
of visual design, to a far more functional user interface never worried th a t the time we spent becoming experts in
technology. I vividly remember sitting in some internet one Microsoft technology or another would be wasted. (In
meetings at Microsoft (this was pre-open-source and be­ future installments of this series of articles, I w ill take a
ing part of these secret closed-door meetings was a big much different Look at this aspect of Microsoft.)
thing for us geeks) where the idea of a DOM (Document
Object Model) was first discussed. How awesome would i t Operating systems were a big thing 20 years ago. Micro­
be to be able to interact with elements on a page using soft's Windows XP is s till one of the most Liked versions
scripting technologies? Mind-boggling! (JavaScript was of Windows (Figure 1). I t was Microsoft's first departure
already a thing back then, of course, but VBScript was from the "battleship gray" user interface design and into
considered equally viable by many). In hindsight, it s now a more colorful world. Most of us w ill fondly remember the
easy to blame Microsoft for having created many non­ default background image of some very green pastures,
standard-compliant HTML problems, but back in the day, overlaid with visual elements th a t featured far more col­
there were no standards and Microsoft had taken i t upon ors than in the past. I remember th a t some of our cus­
themselves to push things forward as fast as possible. tomers thought i t looked "Like a candy store" and i t took
Although I had to suffer many of the later problems this them a while to get used to it. Some even used Windows
caused (and I assume, so did you), I s till think i t was the NT as client operating systems because of it. Ultimately,
right thing to do at the time, and I give them credit for it. Windows XP did become a fan favorite and a very good
operating system for its time.

Other Important Tech I t may have seemed Like that to a Lot of us back then, but
.NET, and Visual Studio were super important technologies i t wasn't ju st a Microsoft world when i t came to operating
for all professional developers. Yes, Linux was also impor­ systems. Linux was always around and important in certain
tant, but when you worked with enterprise customers, scenarios. But there also was this niche operating system
Microsoft was where the lucrative projects were. (Some called Mac OS X. 20 years ago, I thought i t was neat. After
Linux enthusiasts may disagree, and I don't want to take all, i t had been born out of the very geeky-cool NeXT Step
anything away from them, but we weren't successful in operating system, developed by the NeXT company founded
making money with Linux in those days.) Microsoft's anti­ by Steve Jobs, and later integrated into Apple with Steve
trust lawsuit had come to an end, and although i t had an Job's return. I t didn't yet play a big role overall, and most
impact on how Microsoft had to operate for quite some people would never have considered buying a Mac. We used
time to come, i t did solidify Microsoft's position and the Macs in our magazine department, but generally, i t seemed
company represented a very solid and steady bet for most like something that wasn't very important to the business
enterprises around the world. When Microsoft pushed world, and i t was almost non-existent in our software de­
out technology in the early 2000s, you could assume i t velopment considerations. (Another topic I'll have to re­
was going to be not ju s t important and successful in the visit in the next articles in this series.)

Figure 1: A typical Windows XP screen shot

8 CODE: 20 Years Ago codemag.com

Although we b uilt a Lot of web-based applications even
20 years ago, i t should also be mentioned that, for most
businesses, i t was a "thick client" world. Many business
applications were b u ilt as WinForms applications, because
cross-platform deployment wasn't a big consideration for
business applications. After all, why worry about users
on a Mac when nobody in business used Macs? Therefore,
WinForms applications, and Later, WPF applications, were
a very important market segment. (This isn't talked about
very much anymore, but people are often surprised when
I te ll them how many companies are s till building Rich
Windows Client Apps even today.)

Apple was also not a real player in mobile computing yet. Figure 2: The BlackBerry Figure 3: The first Xbox was released for the Christmas
Yes, Apple had the Newton years earlier, but that was 6000, released in 2003 2001 season.
ahead of its time and was soon discontinued. Palm Pilots
were also a thing of the past. But RIM's (Research-In-Mo­
tion) BlackBerry was all the rage for mobile enthusiasts
(Figure 2). I t may have later gotten Hilary Clinton into
trouble as her email device of choice, but it was the state-
of-the-art mobile business solution for quite some time.
I t seems quaint today, but i t was considered unthinkable
that a device without a physical keyboard could be feasible
in business scenarios. This was an idea that Microsoft CEO
Steve Balmer held onto way too long, in the process killing
Microsoft's phone business. Today, most people don't even
remember that Microsoft had a strong position in that mar­
ket segment, with Windows Mobile and Windows CE.

The Fun Stuff

The early 2000s were also a fun time. After all, Microsoft
released the first Xbox (Figure 3). What a machine that was!
And the games it had! I t may not have immediately been
a strong competitor for the already established PlayStation
system, but i t sure put out some classics. Halo, Combat Figure 4: Halo: Combat Evolved
Evolved (Figure 4) was groundbreaking. I t finally made first-
person shooters truly work on consoles and brought such ex­
periences into living rooms. But there were also others. For them Like th a t anymore. Or actually, they do: This was
me, The Elder Scrolls I I I : Morrowind and Star Wars: Knights o f also the start of the superhero movies, an era in which
the Old Republic stand out as particular time sinks. we are s till stuck, i t seems. Additionally, Pirates o f the
Caribbean resonated with people. The Passion o f the Christ
It's hard to believe th a t many of today's (and yesterday's) was released back then, and so was Finding Nemo. Star
biggest brands in computer gaming date back to this era, Wars Episodes I through I I I also fa ll into this time period.
whether you do your gaming on the Xbox, PC, PlaySta­ Yeah. I know. We shall not talk about that.
tion, or elsewhere. I very fondly remember classics such
as Grand Theft Auto I I I , Max Payne, Warcraft I I I , Deus Ex, Markus Egger
Half-Life 2, The Sims, God o f War, and Metal Gear Solid I I , c o d e:
to name ju s t a few. Would you have guessed th a t World o f
Warcraft is now 20 years old? I t seems th a t many of the
games from back then are s till on my "to be played one of
these days soon" list, and not because they are classics,
but ju s t because I haven't quite gotten to them yet.

Although gaming went through a great period, I wasn't as

excited about music back then. Chart toppers from 2003 and
2004 really don't resonate with me as true classics. I guess
50 Cent would disagree and rate In Da Club as one of the all-
time great songs, but I'm not rushing out to buy a remix. The
whole music industry took a blow from the Napster era and
the Apple iPod, released in 2001, hadn't reinvigorated things
yet. Or maybe I just wasn't that into that kind of music.

I did enjoy going to the movies back then though. There

were some great ones. Lord o f the Rings is at the top of my
favorites from 2001 to 2003. Ah... they ju s t don't make

codemag.com CODE: 20 Years Ago 9

 ONLINE QUICK ID 2405021

Async Programming in JavaScript

Some of us like to brag about how old we are because we started working on languages like Assembler or Fortran or Basic.This
was a while ago, when computers were very simple, and although those languages were extremely cryptic and they made us
productive, it's also a reality that we were doing a lot less with them. We were building much shorter buildings. As time moved on,

computers became a Lot more powerful, and our custom­ Let me first start by describing the code you've cloned here.
ers demanded th a t we build skyscrapers. You might have
heard of something called Moore's Law, which is the ob­ Code Structure
servation th a t the number of transistors on an integrated The code you've cloned is a simple nodejs project. I t uses
circuit w ill double every two years with a minimal rise in ExpressJS to serve a static website from the public folder,
cost. Yes, computers have become very powerful, but ju s t as can be seen in Figure 1.
due to basic physics and energy density issues, we've also
h it a wall in absolute computing power th a t we can pack To run it, just follow the instructions in readme.md. At a
within a single processor. As a result, the world started high level, it's a matter of running npm install, and hitting
Sahil Malik moving toward multiple cores, and multiple processors; F5 in VSCode. Additionally, you'll see that in index.js as seen
www.winsmarts.com even your phone— or maybe even your watch— now has in Listing 1, in addition to serving the public folder as a
@sahilmaLik multiple cores or multiple processors inside it. static website, I'm also exposing an API at the "/random"
URL. This API is quite simple; it waits for five seconds and
Sahil Malik is a Microsoft
When these multiple cores or multiple processes try to work returns a random number. I have a wait here to demonstrate
MVP, INETA speaker,
together, adding two processors doesn't always equal 2X the what effect blocking processes, such as this wait, can have
a .NET author, consultant,
performance. Sometimes it can even be less than IX because your browser's UI. The reason I've written this code in Node-
and trainer.
the competing processors might be working against each JS is because I could use identical code for the wait on both
Sahil loves interacting other. For sure, the benefit you get w ill be less than 2X be­ client and server, although this isn't a hard requirement.
with fellow geeks in real cause some overhead is spent on coordination. Now imagine
time. His talks and train­ if you have a 64-core processor, how would that look? And Let's also briefly examine the client side code. The in-
ings are full of humor and how would you write code for it? There w ill always be that dex.html file is quite simple. I t references jQuery to help
practical nuggets. one really smart guy on your team that understands the dif­ simplify some of the JavaScript I 'll write. I t has a but­
ference between mutexes and semaphores, and that smart ton called btnRandom th a t calls a JavaScript. I t has a
His areas of expertise are guy w ill act Like the cow that gives one can of milk and tips div called "output" where the JavaScript can show mes­
cross-platform Mobile app over two. His smarts w ill make rest of the team unproduc­ sages to communicate to the user. The idea is th a t I 'll call
development, Microsoft tive because, let's be honest, these concepts can be hard to a function th a t blocks for five seconds, and I'LL show a
anything, and security
understand, harder to write, and very hard to debug. "start" and the random number output message when the
and identity.
function starts and then when it's done.
Although it's no surprise th a t as we're building more
S ^ E complex software, our platforms and languages have also Additionally, I've placed a text area where users can type
ex- r^:? evolved to help us deal with this complexity, so the entire freely. The function takes five seconds to complete, so
team of mere mortals is productive. Languages have also what I'd like you to try is, within those five seconds, try
evolved to support more complex paradigms, and JavaScript to type in th a t text area. I f you can type in th a t text area
is no exception. while the function is executing, that's a non-blocking UI,
which is a good user interface. But i f the UI is frozen and
In this article, I'm going to explore a back-to-basics ap­ you cannot type in th a t textbox while your function runs,
proach by explaining asynchronous programming in Ja­ that's a bad user experience.
vaScript. Let's get started.
The user interface of my simple HTML file looks Like Fig­
ure 2. The index.html file can be seen in Listing 2.
A Little Pretext
Before I get started, there's a little challenge I must deal
with. Demonstrating asynchronous concepts through text A Synchronous Call
and images as they appear in this article can be difficult. So Let's first start by writing a simple JavaScript function that
I'm going to describe the various concepts, but you should takes five seconds to execute. At the end of five seconds, it
also grab the associated code for this article at the follow­ simply returns a random number. This function is basically
ing URL: https://github.com/maliksahil/asyncjavascript. the same function you see in index.js called "waitForMil-
I recommend running the code side-by-side as you read this LiSeconds", except that for now, I'll ju st run it client side,
article as that w ill help cement the concepts. and the function itself w ill return a random number.

The function is called on the click event of the button you

Press button to make async call: | Click see in Figure 2. As soon as the user clicks on the button,
i t shows a "Start" message in the output div. Then the
T ry ty p in g here w h ile th e long running function runs for five seconds and five seconds later, you
c a l l is running . .
should see a random number shown in the output div. The
Figure 1: The project code for the synchronous call is refenced in index.html as
structure Figure 2: The user interface scripts/l.sync.js and can be seen in Listing 3.

10 Async Programming in JavaScript codemag.com

Go ahead and run npm start (or F5 in VSCode) and visit Listing 1: The index.js server side file
the browser at http://localhost:3000. Click the "Click"
button from Figure 2, and immediately try typing in the const express = require(' express' );
const app = expressO;
text area below. What do you see? const PORT = 3000;

You'Ll notice that until the call completes and the random app.use(express.static( 1public1));
app.get("/random", (request, response) => {
number is shown in the output div, the page is essentially wai tForMiIT i Seconds(5000);
frozen. I t accepts no input from the user. In fact, the page is const random = {
dead: I t accepts or responds to no events. This is certainly a "random": Math .floor (Math. random() * 100)
bad user experience but may also lead to inexplicable bugs. };
response.send(random);
});
Callbacks app.listen(PORT, () =>
console.log('Server listening on port: ${PORT}'));
Now let's explore a technique in JavaScript called callbacks.
I f you remember what you first did, this line stands out: function waitForMilliSeconds(milliSeconds) {
var date = new Date();
var curDate = null;
randomNum = waitForMiHi Seconds(5000); do { curDate = new Date(); }
while (curDate - date < milliseconds);
This means th a t the returnvalue of waitForMilliSeconds is }
what gets populated in randomNum.

We've learned from other languages th a t you could pass in Listing 2: index.html
a function pointer to waitForMilliSeconds. Wouldn't i t be <html>
nice i f waitForMilliSeconds could call th a t function point­
er when it's done with its five seconds of blocking work? <head>
<scri pt
src="https://code.jquery.com/jquery-3.7.1.min.j s"
To facilitate that, modify your waitForMilliSeconds function, i ntegri ty=".."
as shown below in the next snippet. The Login has been crossorigin="anonymous"></script>
</head>
trimmed for brevity and the only change is that instead of
sending back a return value, you're now accepting a param­ <body>
eter called callbackFunc. When you're done with your work, Press button to make async call:
<button type="button" id="btnRandom">Click</button>
you simply call this callback function and pass in the result. <br />
<div id="output"></div>
function waitForMiHiSeconds( <script src="scripts/l.sync.js "></script>
milliseconds, callbackFunc) { <br/>
var date = new Date(); <textarea rows="5" cols="40">Try typing here
while the long running call is running . .</textarea>
random = Math .floor (Math. random() * 100); </body>
callbackFunc(random); </html>
}

Accordingly, how you call this method also changes. This Listing 3: l.sync.js client side synchronous JS
can be seen below.
$("#btnRandom").on("click", function () {
$("#output").text("Start");
waitForMiHi Seconds(5000, (random) => { randomNum = waitForMiHi Seconds(5000);
$("#output").text(random); $("#output").text(randomNum);
}); });
function waitForMilliSeconds(miHiSeconds) {
As you can see, you're now calling waitForMilliSeconds var date = new Date();
with two input parameters. The firs t parameter instructs var curDate = null;
do { curDate = new Date(); }
the function to wait for five seconds and the second is an while (curDate - date < milliseconds);
anonymous function parameter. This function gets called return Math .floor (Math. random() * 100);
once waitForMilliSeconds is done and i t calls the callback- }
Func variable function.

Before you run this, what do you expect the behavior will Promise
be? Will it block the UI or not? Let's find out. Go ahead and JavaScript has yet another way of structuring your code,
run this. You'll notice that although the code seems to have which is Promises. You may have seen them when w rit­
a different structure, the callback seemed to have no effect ing AJAX code, where your code can make an HTTP call
on the single-threaded nature of the code. The UI still blocks. to the server w ithout refreshing the whole page. This is
how complex apps such as Google Maps were born. Before
Bummer! Google Maps, navigating a map required you to refresh
the whole page. I t was a horrible user experience, until
Well, at Least you Learned a new concept here, and th a t someone showed us a better way. Technically speaking,
such callbacks have no effect on the single-threaded na­ Outlook for the web was leveraging this technique al­
ture of execution. ready, but hey, this isn't a race.

codemag.com Async Programming in JavaScript 11

 Listing 4: Using Promises A w ful! Promises and Lies. I guess th a t d id n 't solve the
problem either.
function wai tForMi Hi Seconds(mi Hi Seconds) {
const myPromise = new Promise((resolve, reject) => {
var date = new Date();
XHR
random = Math.floor (Math. randomQ * 100);
resolve(random); A t th is p o in t, you must be th in k in g th a t you've w ritte n
}); so much AJAX code, and th a t code Leveraged Promises,
return myPromise; callbacks, and other paradigms. For sure you d id n 't see
} th a t blocking behavior there. W hat is so special about
AJAX th a t i t doesn't block?

Listing 5: Simple XHR request There are many ways to w rite AJAX code. I've referenced
$("#btnRandom").on("click", function () { jQuery and certainly jQuery has abstractions th a t help w rite
$("#output").text("Start"); AJAX code. The most basic way to call AJAX is by using XHR.
const xhr = new XMLHttpRequest();
xhr.addEventListener("loadend", () => { The way XHR works is th a t you in s ta n tia te a new instance
$("#output").text(xhr.responseText); o f XMLHttpRequest. You subscribe to the loadend event
}); and fire your HTTP request. Now whenever the call re­
xhr.open("GET", "/random"); turns, the loadend event gets called and you get the re­
sults. You can process accordingly w hether it's an error or
xhr.send();
$("#output").text("Sent xhr request"); success.
});
Let's s ta rt by in s ta n tia tin g the XMLHttpRequest.

Listing 6: XHR and Promises const xhr = new XMLHttpRequest();

function makeXhrCall() {
const myPromise = new Promise((resolve, reject) => { Before you send the request, Let's subscribe to the loa­
const xhr = new XMLHttpRequest(); dend event. You're going to call an anonymous method
xhr.addEventListener("loadend", () => {
resolve(xhr.responseText); when the event gets called, which shows whatever re­
}); sponse the server sent.
xhr.open("GET", "/random");
xhr.send();
}); xhr.addEventListener("loadend", () => {
return myPromise; $("#output").text(xhr.responseText);
} });

Next, let's send the request.

There's also pure client-side code. W hat i f you were to use
Promises instead o f callbacks. W ill the code n o t be single xhr.open("GET", "/random");
threaded then? Let's fin d out. xhr.sendQ;

The idea behind a JavaScript Promise is th a t the func­ You can see the fin a l code th a t puts a ll th is to g ethe r in
tio n doesn't return a value, b u t instead returns a Promise. Listing 5.
The Promise w ill e ith e r resolve (succeed) or reject (fa il).
When i t resolves, i t can send back a success o u tp u t. I f i t Remember from Listing 1, the server-side code is basical­
fails, i t can send back an error. ly the same code you've been using except now instead o f
running on the clie n t, it's running on the server. I t waits
Your caller then uses standard paradigms around Promises five seconds and sends back a random number.
to handle success w ith a Then method.
Now go ahead and run th is by referencing th is script, press­
Let's m odify the waitForM illiSeconds method to now re­ ing F5 to refresh the browser, and clicking the button.
turn a Pomise and resolve i t on success. You can see th is
method in Listing 4. Very in terestingly, now the UI doesn't block. How odd is
that?
This allows you to w rite calling code:
A lthough th is is great, w o u ld n 't i t be nice i f complex c li­
waitForMiHi Seconds(5000).then( (random) => { ent-side code could be afforded the luxury o f being m u lti­
$("#output").text(random); threaded? This XHR-based code feels so com plicated. My
}); example was simple, b u t im agine how th is could Look w ith
m u ltip le dependencies, in p u ts dependent on other XHR
Let's run th is code again and h it the click b u tto n . What calls succeeded, tim in g issues, etc. Ugh!
do you see?

Ah! Yet again, although the code functionally is accurate, i t Promises and XHR
s till blocks the UI thread. The code is s till single threaded. I t Let's s ta rt by cleaning th is code up a b it. You've already
responds to no in p u t while the function is running and sud­ seen Promises in action. Can you combine XHR and Prom­
denly reacts to key strokes queued up in those five seconds. ises tog ethe r to help w rite code th a t's simpler? Sure! ALL

12 Async Programming in JavaScript codemag.com

you'd have to do is abstract o u t a ll th is XHR code in to
its own method th a t returns a Promise. When you do an
xhr.send(), ju s t return the Promise. When XHR's loadend
event is called, e ith e r resolve or reject the Promise as per
the return results. This can be seen in Listing 6.

By doing so, the calling code becomes a lo t simpler, as

dtSearch
can be seen below.

makeXhrCaH.().then((output) => {
Instantly Search
});
$("#output").text(output);
Terabytes
Now go ahead and run th is code. I t runs ju s t Like before
and i t doesn't block the UI thread. Is th is because you're
using a Promise or th a t you're using an XHR? Well, you
did use a Promise on a loop th a t was en tire ly on the c li­
en t side and th a t did block the UI. This non-U I blocking
magic is b u ilt in to XHR. dtSearch's docum ent filters support:
• popular file types
Async Await • emails with multilevel attachments
Recently, JavaScript introduced support fo r async aw ait
keywords. The Promise code looks cleaner than pure XHR
• a wide variety of databases
code, b u t i t s t ill feels a b it inside out. Im agine i f you had • web data
three Promises you needed to w a it fo r and those inputs
go in to tw o more Promises, which fin a lly go in to another
AJAX call? Luckily, Promises do have concepts such as
resolveALl etc., which do help. They're an im provem ent
over pure XHR code. However, the code becomes severely
Over 25 search options including:
indented and you're stuck in a hell hole o f brace m atching • efficient multithreaded search
and keeping your code under 80 characters w idth.
• easy multicolor hit-highlighting
Async await helps you tackle th a t problem. Look a t the sync • forensics options like credit card search
code example from Listing 3. To convert th a t from sync to
async, all you have to do is add the async keyword in fron t
o f the function. In other words, change this line o f code:

function waitForMilliSeconds(milliseconds) { Developers:

in to th is line o f code:
• SDKs for Windows, Linux, macOS
• Cross-platform APIs cover C++, Java
async function waitForMilliSeconds(milliSeconds) { and current.NET
That's it ! Okay, I lied. Your calling pattern changes sligh tly • FAQs on faceted search, granular data
too, b u t fo r the better. Your calling code changes like this: classification, Azure, AWS and more
$("#btnRandom"),on("click", async () => {
$("#output").text("Start");
random =
await wai tForMi Hi Seconds (5000);
$("#output").text(random);
});
Visit dtSearch.com for
• hundreds of reviews and case studies
The changes aren't severe a t all. A ll you had to do was
fo llo w tw o rules: • fully-functional enterprise and
developer evaluations
• Put "a w a it" in fro n t o f a method call th a t is in ­
tended to be async.
• You cannot use aw ait in any method th a t is n 't async The Smart Choice for Text
its e lf.
Retrieval® since 1991
Now you can fin a lly s ta rt w ritin g async code th a t doesn't
look like a severely indented case o f brace-m atching dis­ dtSearch.com 1-800-IT-FINDS
ease. You can extrapolate th is to an XHR example, as can
be seen in Listing 7.

codemag.com Async Programming in JavaScript 13

 Listing 7: Async XHR calls As you w ill see shortly, you have fu ll co n tro l on the "d a ta "
property and you can define your own structure. Let's get
let makeXhrCall = async () => { to th a t in a moment. First let's focus on w hat "w aitF orM il-
const myPromise = new Promise((resolve, reject) => {
const xhr = new XMLHttpRequest(); liSeconds" looks Like in th is worker world.
xhr.addEventListener("loadend", () => {
resolve(xhr.responseText);
Below is an abbreviated version o f waitForM illiSeconds
});
xhr.open("GET", "/random"); th a t communicates back to the caller via the postMessage
xhr.sendQ; method.
});
return myPromise;
} function wai tForMi Hi Seconds (mi Hi Seconds) {

random = Math.floor (Math. random() * 100);

Listing 8: Async A w ait w ith Workers postMessage(random);
const worker = new Worker( '. /scripts/hardwork.js'); }
$("#btnRandom").on("click", async () => { I removed the actual logic for brevity, but it's the same w ait­
$("#output").text("Start");
random = await doHardWork(); ForMilliSeconds function I've used numerous times in this
$("#output") .text(random); article already. That's it. This is what the worker looks like.
});
async function doHardWork() { Now let's see how calling th is worker looks.
return new Promise((resolve) => {
worker.postMessage({
command: "Start", "milliseconds": 5000 }); As can be seen in Listing 8, you firs t start by creating a
worker.addEventListener("message", (output) => { worker object using the "hardw ork.js" file . Then, in the
resolve(output.data); doHardWork method, post a message w ith a custom ob­
je c t structure, which is made available as the message.data
property to the worker. I th in k "add a listener" for "mes­
sage" and whenever a message is available, which is after
the five second loop, I grab the random number o u tp u t and
SPONSORED SIDEBAR Now when you run th is code, although it's quite s im p li­ resolve it. I th in k to use the async aw ait pattern to set the
fied, unless it's an XHR call, i t s t ill seems to block the UI. o u tp u t div's value.
Adding Copilots
toYour Apps Async Await with Workers
Wait a minute. This looks quite sim ilar to the XHR object,
doesn't it? Over there also, you had an addEventListener,
The fu tu re is here no w What's so peculiar about XHR th a t i t doesn't block the only the actual event was different.
and you don't want to get UI thread? I t seems to work on an Eventing model.
left behind. Unlock the Luckily fo r you, you can leverage th a t same cap ab ility Go ahead and run this code. Remember th a t this is a client­
tru e p o te n tia l of your using workers in JavaScript. Workers in JavaScript is a side loop. Go ahead and click the button. The code behaves as
software applications by to p ic in its e lf, b u t fo r my purposes here, you need to before, but now the UI thread no longer locks. The te xt area
adding Copilots. separate o u t the blocking client-side code in its own remains responsive while the loop is running w ithout XHR.
worker, which, in th is case, means its own JavaScript
CODE C onsulting can
file . Great. You've fin a lly achieved the panacea o f clean code,
assess your applications
th a t doesn't block the UI thread.
and provide you with
a roadmap for adding This worker w ill now listen for, and respond to , messages
a fte r the work is done. So go ahead and create a new file
Copilot features and
optionally assist you in under the Scripts fold er called hardwork.js. Here's where Summary
adding them to your you in te nd to run your hard-working loop th a t takes five As computers become increasingly pow erful and complex,
applications. seconds to complete. it's reasonable to assume th a t we're going to have to w rite
increasingly com plicated code. To w rite th a t com plicated
Reach o u t to us today Inside th is hardwork.js, you fir s t need to add an event code, we're going to have to Learn new patterns, such as
to get your application listener fo r "message". This looks like: the asynchronous patterns available in JavaScript.
assessment scheduled.
www.codemag.com/ai addEventListener("message", (message) => { In th is article, I discussed many such patterns, and I b u ilt
if (message.data.command === "Start") { a story b it by b it to show you how you can use the v a ri­
wai tForMi Hi Seconds ( ous paradigms in JavaScript to create non-blocking code
message.data.mini Seconds) th a t's easy to m aintain.
}
}); There's a lo t more to Learn, o f course, and I'm sure, as
tim e moves forward, greater demands and b e tte r patterns
As you can see, you're adding an event Listener fo r "mes­ w ill emerge.
sage". Whenever th is is called, you can assume th a t some
in p u t parameters are sent to you, in th is case, via "mes- This is the beauty o f our industry. Never a boring day.
sage.data.command", which te lls you w hat action to take.
The code is p re tty simple, so you ju s t Look fo r "S tart". U n til next tim e, happy coding!
A dditionally, the data ob je ct has another property called
m illiseconds th a t te lls you how Long to w a it before re­ Sa h it Malik
tu rn in g a random number. CODE

14 Async Program m ing in JavaScript codemag.com

Copilot • Al • GitHub • .NET • Visual Studio • C# • Azure Al • Blazor • ChatGPT • ASP.NET • OpenAI • a

S E P T 1 0 -1 2 WORKSHOPS SEPT 8,9 & 13

MGM GRAND, LAS VEGAS, NEVADA
Let the Experts Guide You on Your A lJourney:

SCOTT HANSELMAN SCOTT HUNTER MILAN KAUR DAN WAHLIN HEATHER DOWNING
Vice President, Vice President, Director Product Manager, Principal Cloud International Speaker &
Developer Community, Product Management Microsoft Developer Advocate, Developer Advocate
Microsoft for Azure Developer Microsoft
Experience, Microsoft

JEFF FRITZ JOHN PAPA MARKUS EGGER ZOIN ER TEJADA MICHELE LEROUX
Principal Program Partner G M - Cloud President and Chief CEO and Architect, BUSTAMANTE
Manager, Microsoft Advocacy, Microsoft Software Architect, Solliance President & Architect,
CODE Group Solliance

REGISTER TODAY!
w DEVintersection.com X@DEVintersection
Q [email protected]

BONUS: Al for Decision Makers Track

Questions Answered, Strategies Delivered, Relationships Built

 ONLINE QUICK ID 2405031

Manipulating JSON Documents

in .NET 8
JavaScript Object Notation (JSON) is a great way of storing configuration settings for a .NET application. JSON is also an efficient
method to transfer data from one machine to another. JSON is easy to create, is human readable, and is easy for programming
languages to parse and generate. This text-based format for representing data is language agnostic and thus easy to use in C#,

JavaScript, Python, and alm ost any programming Lan­ Nested Objects
guage existing today. In th is article, you're going to Learn Each value after the name can be one of the various data
m ultiple methods o f creating and m anipulating JSON doc­ types shown in Table 1. Although this isn't a large List of data
uments in .NET 8. In a d d itio n , y o u 'll Learn how to serialize types, all data can be expressed w ith ju s t this set o f types.
and deserialize C# objects to and from JSON.
Look a t Figure 4 to see an example o f a JSON object th a t
has a value o f each o f the data types fo r each name. The
JSON Structure "name" property has a string value "John Sm ith" enclosed
As shown in Figure 1, a JSON o b je ct is made up o f a w ith in double quotes. The "age" property is a numeric w ith
Paul D. S h e riff collection o f nam e/value pairs. You may hear these also a value o f 31. The "ssn" property is empty as represented
http://www.pdsa.com expressed as key/value pairs, or, in C# terms, th is is a by the keyword null. The "address" property is another
property and a value. In C#, a JSON ob je ct is the equiva­ JSON object and is thus enclosed by curly braces. The
Paul has been working le n t o f an object, record, or a stru ct w ith property names, "phoneNumbers" property is a JSON array, w ith each value
in the IT industry since and the values you assign to those properties. JSON can o f the array another JSON object. As you see, JSON is very
1985. In that time, also be a collection o f one or more objects (Figure 2 flexible and can represent alm ost any type o f structure you
he has successfully
and Figure 3 ). In C#, th is would be the equivalent o f a can possibly need in your programming tasks.
assisted hundreds of
dictionary, hash table, keyed List, or associative array. A l­
companies'architect
though I'm going to use C# in th is article, the constructs
software applications
m entioned are universal across a ll modern programming JSON Manipulation Classes
to solve their toughest
Languages. Because o f th is , any Language can m anipulate There are several classes w ithin a couple of namespaces in
business problems. Paul
has been a teacher and these JSON objects easily. .NET 8 th a t you use to work with JSON in your C# applications.
mentor through various The first namespace is System.Text.Json and the second
mediums such as video A JSON ob je ct begins w ith a Left brace and ends w ith namespace is System.TextJson.Nodes. You have most likely
courses, blogs, articles and a rig h t brace (see Figure 1). Each name is follow ed already used the JsonSerializer class to serialize C# objects to
speaking engagements by a colon and the nam e/value pairs are separated by JSON, or to deserialize a JSON string into a C# object. Howev­
at user groups and a comma. Each name must be wrapped w ith in double er, there are other classes you can use to add nodes to a JSON
conferences around the quotes. A ll string values must be wrapped w ith in double object, set and retrieve values, and create new JSON objects.
world. Paul has multiple quotes.
courses in the www. The System.TextJson Namespace
pluralsight.com library A JSON array is an ordered collection o f values th a t be­ W ithin th is namespace are classes and structures to help
(https : // b it . Ly/3gvXgvj) gins w ith a le ft bracket and ends w ith a rig h t bracket. you m anipulate JSON documents in clu din g the JsonSe-
and on Udemy.com Each value w ith in the array is separated by a comma. The rializer class. Table 2 provides a description o f each o f
(https://bit.Ly/3W 0K8kX) values w ith in the array may be a single ite m , such as a the classes w ith in th is namespace. Each o f these classes
on topics ranging from C#, string or a number (Figure 2 ), or each value w ith in the is illu s tra te d th ro u g h o u t th is article. A ll the classes and
LINQ, JavaScript, Angular, array may be a JSON object (Figure 3 ). structures w ith in th is namespace are im m utable. This
MVC, WPF, XML, jQuery,
and Bootstrap. Contact
Paul at [email protected].

Figure 1: JSON objects are made up o f nam e/value pairs.

Figure 2: JSON arrays can have ju s t simple values as Figure 3: Each elem ent in a JSON array can be a JSON
th e ir elements. object.

16 Manipulating JSON Documents in .NET 8 codemag.com

 Data Type Description
Number A signed number value. The number may be a whole number (integer) or a decimal w ith a fractional part.
String A set of zero or more Unicode characters enclosed w ithin double quotes. Strings support a backslash escaping syntax ju s t as you find in C#.
Boolean Either a true or a false value.
Object A JSON object using the JSON object syntax previously explained.
Array A set of zero or more values using the JSON array syntax previously explained. Each element w ithin the array may be of any of the types
shown in this table.
Null Use the keyword null to signify an empty value for this name.

Table 1: JSON has a Limited set o f data types available fo r a value.

Class / Structure Description

JsonDocument A class th a t represents an immutable (read-only) document object model (DOM) of a JSON object. Use this class when you don't
have a C# class to deserialize the JSON in to and you need to access the name/value pairs programmatically.
JsonProperty This structure represents a single JSON property w ithin a JSON object. For example, "colorld": 1 is an example of a JsonProperty.
JsonELement This structure represents a single value w ithin a JSON property. For example, the number one (1) w ithin the property "colorld": 1
is the JsonELement.
JsonSerializer A class used to serialize a JSON string in to a C# object, or to deserialize a C# object in to a JSON string.
Utf8JsonWriter A class th a t can be used to em it a JSON document one property at a time. This class is a high-performance, forward-only, non­
cached method of w riting UTF-8 encoded JSON text.

Table 2: The System .TextJson namespace contains classes and structures fo r m anipulating and serializing JSON objects

Class Description
JsonObject This class represents a mutable (read/write) JSON document. This class is Like the JsonDocument class from the System.TextJson namespace.
JsonNode This class represents a mutable (read/write) node w ithin a JSON document. This class is like the JsonProperty class from the System.TextJson
namespace.
JsonValue This class represents a mutable (read/write) JSON value. This class is like the JsonELement class from the System.TextJson namespace.
JsonArray This class represents a mutable (read/write) JSON array.

Table 3: The System.TextJson.Nodes namespace contains classes fo r m anipulating in-m em ory JSON objects as a docum ent ob je ct model

means th a t once they've been in sta n tia te d w ith data, Below the Using statement, create a variable named jo th a t is
they cannot be m odified in any way. an instance of a JsonObject. The JsonObject provides the abil­
ity to add, edit, and delete nodes w ithin the JSON document.
The System.TextJson.Nodes Namespace
The classes in th is namespace (Table 3) are for creating JsonObject jo = new();
and m anipulating in-memory JSON documents using a DOM.
These classes provide random access to JSON elements, al­
low adding, editing, and deleting elements, and can con­
vert dictionary and key value pairs in to JSON documents.

Build In-Memory JSON Documents

I recommend fo llo w in g along w ith th is article to ensure
th a t you have a solid foundation fo r m anipulating JSON
documents. The too ls needed to fo llo w the step-by-step
instructions in th is article are .NET 6, 7, or 8, and e ith e r
Visual Studio 2022 or VS Code. Create a console applica­
tio n named JsonSamples. The goal o f th is fir s t example
is to create the fo llo w in g JSON object.

{
"name": "John Smith",
"age": 31
}

Of course, there are many ways to create th is JSON object

using C# and the JSON classes. For th is firs t example, open
the Program.es file , delete any lines o f code in th is file ,
and add the follow ing Using statem ent as the firs t Line.

using System.TextJson.Nodes; Figure 4: Each elem ent in a JSON object can be another object, or an array.

codemag.com Manipulating JSON Documents in .NET 8 17

 Add two name/vaLue pairs to the JsonObject using the C# };
KeyValuePair class, as shown in the following code.
Console.WriteLine(jo.ToString());
jo.Add(new KeyValuePair<string,
JsonNode?>("name", "John Smith")); Because the JsonObject can be initialized with a Diction­
ary object, you may use the same syntax you used to cre­
jo.Add(new KeyValuePair<string, ate a Dictionary object in the constructor of the JsonOb­
JsonNode?>("age", 31)); ject, as shown in the following code:

Write out the JSON document using the ToString() meth­ using System.Text.Json.Nodes;
od on the JsonObject.
JsonObject jo = new() {
Console.Wri teLi ne(jo.ToString()); ["name"] = "John Smith",
["age"] = 1
The output from this statement is the JSON object shown };
earlier. Notice how the JSON is nicely formatted. I f you
wish to remove all of the carriage returns, Line feeds, and Console.WriteLine(jo.ToString());
whitespace between all the characters, change the call
from the ToString() method to the ToJsonString() method Create Nested JSON Objects
instead. You should then see the following JSON appear Not all JSON objects are simple name/value pairs. Some­
in the console window. times you need one of the properties to be another JSON
object. The "address" property is another JSON object
{"name":"John Smith","age":31} th a t has its own set of name/value pairs, as shown in the
following snippet:
Use a New C# 12 Feature
In C# 12 (.NET 8) you can create the JsonObject using the {
following syntax. Note that the square brackets used in the "name": "John Smith",
code are a new C# 12 feature that allows you to initialize "age": "31",
the new JsonObject object without using the new keyword. "ssn": null,
"isActive": true,
JsonObject jo = "address": {
"street": "1 Main Street",
new KeyValuePair<string, "city": "Nashville",
JsonNode?>("name", "John Smith"), "stateProvince": "TN",
new KeyValuePair<string, "postalCode": "37011"
JsonNode?>("age", 31) }
]; }

Using a Dictionary Class To create the above JSON object, create a new instance
You can pass an instance of a Dictionary<string, Json- of a JsonObject and, using a Dictionary object, build the
Node?> to the constructor of the JsonObject to create structure you need, as shown in the following code snip­
your JSON document. Replace the code in the Program.es pet:
file with the following:
using System.Text.Json.Nodes;
using System.Text.Json.Nodes;
JsonObject jo = new() {
Dictionary<string, JsonNode?> diet = new() { ["customer"] = "Acme",
["name"] = "John Smith", ["IsActive"] = true,
["age"] = 31 ["address"] = new JsonObjectQ {
}; ["street"] = "123 Main Street",
["city"] = "Walla Walla",
JsonObject jo = new(dict); ["stateProvince"] = "WA",
["postalCode"] = "99362",
Console.WriteLine(jo.ToString()); ["country"] = "USA"
}
Using a JsonValue Object };
The Add() method on the JsonObject class also allows you
to pass in the name and a JsonValue object. Pass in the Console.WriteLine(jo.ToString());
value to static method Create() on the JsonValue class to
create a new JsonValue object.
Parse JSON Strings into Objects
using System.Text.Json.Nodes; JSON documents are commonly stored as strings in a file or
in memory. Instead of attempting to read specific values in
JsonObject jo = new() { the JSON using File 10 or string parsing, you can parse the
{ "name", JsonValue.Create("John Smith") }, string into a JsonNode object. Once in this object, it's very
{ "age", JsonValue.Create(31) } easy to retrieve single values, or entire nodes.

18 Manipulating JSON Documents in .NET 8 codemag.com

 Listing 1: To save typing, I have created a class with some JSON documents
namespace JsonSamples; postalCode"": ""37011

public class JsonStrings

{
public const string PERSON =
@"{ public const string PER50N_ADDRESS_PH0NES =
""name"": ""John Smith"",
""age"": 31, ""name"": ""John Smith"",
""ssn"": null, ""age"": 31,
""isActive"": true ""ssn"": null,
}"; ""isActive"": true,
""address"": {
public const string PHONE_NUMBERS = ""street"": ""1 Main Street"",
@"[ ""city"": ""Nashville"",
{ ""state""' ""TN""
""type"": ""Home"", ""postalCode"": ""37011""
""number"": ""615.123.4567"" }.
}. ""phoneNumbers"": [
{ {
""type"": ""Mobile"", ""type"": ""Home"",
""number"": ""615.345.6789"" ""number"": ""615.123.4567""
}, }.
{ {
""type"": ""Work"", ""type"": ""Mobile"",
""number"": ""615.987.6543"" ""number"": ""615.345.6789""
} }-
]"; {
""type"": ""Work"",
public const string PERSON_ADDRESS = ""number"": ""615.987.6543""
}
""name"": ""John Smith"", ]
""age"": 31, }";
""ssn"": null,
""isActive"": true,
""address"": {
""street"": ""1 Main Street"",
""city"": ""Nashville"",
""state"": ""TN"",

Instead of repeating the same set of JSON strings

throughout this article, I'm creating a single class (List­ "name": "John Smith",
ing 1) with some string constants to represent differ­ "age": 31,
ent JSON documents. The PERSON constant is a JSON "ssn": null,
document th a t contains four properties to represent a "isActive": true
single person. The PHONE_NUMBERS constant is a JSON }
array of a few phone numbers where each number has Object
a type property with a value such as Home, Mobile, or
Work. The PERSON_ADDRESS constant contains a nested The firs t Console.WriteLine() statement emits the JSON
address property th a t is another object with a street, object, and the second Console.WriteLine() statement
city, state, and postalCode properties. The PERSON_AD- reports the kind of object contained in the JsonNode
DRESS_PHONES constant contains person information, object that's the type of Object. Of course, you may
address information, and an array of phone numbers all pass to the Parse() method any of the other constant
in one JSON object. strings. Write the following code in the Program.es file
to parse the PHONE_NUMBERS constant into a JsonNode
Create a JsonNode Object object:
The JsonNode class is probably the class you w ill use most
often. It's very flexible and contains most of the function­ using JsonSamples;
ality you need when manipulating JSON documents. For using System.Text.Json.Nodes;
example, to parse the PERSON constant into a JsonNode
object, place the following code into the Program.es file: // Parse string into a JsonNode object
JsonNode? jn = JsonNode.Parse(
using JsonSamples; JsonStrings.PHONE_NUMBERS);
using System.Text.Json.Nodes;
Console.WriteLine(j n?.ToString());
// Parse string into a JsonNode object Console.Wri teLi ne(j n! . GetValueKi nd());
JsonNode? jn = JsonNode.Parse(JsonStrings.PERSON);
Run the console application and you should see the fo l­
Console.Wri teLi ne(j n! .ToStri ng()); lowing displayed in the console window:
Console.Wri teLi ne(j n! . GetValueKi nd());

Run the console application and you should see the fo l­

lowing displayed in the console window: "type": "Home",

codemag.com Manipulating JSON Documents in .NET 8 19

 "number": "615.123.4567" {
}. "name": "John Smith",
{ "age": 31,
"type": "Mobile", "ssn": null,
"number": "615.345.6789" "isActive": true
}. }
{ Object
"type": "Work",
"number": "615.987.6543"
}
Read Data from JSON Documents
There are a few d iffe re n t ways you can read in d ivid u a l
Array nam e/value pairs from a JSON document. Both the Json-
ELement and the JsonNode classes allow you to get a t the
Notice th a t in th is case, the GetValueKind() method re­ data w ith in the JSON.
ports th is as an Array instead o f an Object. When the
JSON string th a t is read in starts w ith a square bracket, Once you've parsed some JSON in to a JsonDocument ob­
i t s a JSON array instead o f a JSON object. je c t, you must always use the RootElement property to
retrieve specific values w ith in the JSON docum ent. You
Create a JsonDocument Object can e ith e r place the RootElement property in to a Json-
Earlier in th is article, you Learned th a t you could add item s ELement object, or you can use the fu ll path o f the Json-
to a JsonObject in its constructor. There's no constructor Document.RootElement property. W rite the code shown
fo r the JsonDocument object, so you must use the Parse() in Listing 2 in to the Program.es file .
method to get valid JSON data in to th is object. The Json­
Document object is a very e ffic ie n t object to use when In Listing 2, you parse the PERSON string in to a Json­
a ll you need to do is to read data from a JSON document. Document. You then get the property called "name"
Once the data is parsed in to the JsonDocument, access from the JSON docum ent and place th is in to a JsonEle­
the RootElement property to retrieve the JSON. W rite the m ent ob je ct named je . Because the "name" property is a
fo llo w in g code in to the Program.es file : string value, call the G etString() method on the je v a ri­
able to extract the value from the "name" property. I f
using JsonSamples; you d o n 't wish to use a separate variable, you may ac­
using System.Text.Json; cess the jd.RootELement property d ire ctly by calling the
GetPropertyC'age") to get the "age" elem ent. Call the
// Parse string into a JsonDocument object G e tln t3 2 () method on th is elem ent to extract the "age"
using JsonDocument jd = JsonDocument.Parse(JsonStrings.PERSON); value and display i t on the console. Run the application
and you should see the fo llo w in g o u tp u t from Listing 2
// Get Root JsonElement structure displayed in the console window:
JsonElement je = jd.RootElement;
Name=John Smith
Console.WriteLine(je.ToString()); Age=31
Console.Wri teLi ne(j e.ValueKind);
Retrieve Data in a Nested Object
In the code above, you retrieve the RootElement prop­ Look back a t Listing 1 and notice th a t the PERSON_AD-
erty and place i t in to a new variable o f the type Json- DRESS constant is the one w ith the nested "address" ob­
ELement. It's th is class th a t you're going to use to read je c t. To access the "c ity " value w ith in the "address", re­
the data from JSON document, as you shall soon learn. place the code in the Program.es file w ith the fo llo w in g :
Run the application and you should see the fo llo w in g dis­
played in the console window: using JsonSamples;
using System.Text.Json;

Listing 2: Retrieve values from the JSON using the RootElement property / / Parse string into a JsonDocument object
using JsonSamples; using JsonDocument jd =
using System.Text.Json; JsonDocument.Parse(JsonStrings.PERS0N_ADDRES5);

/ / Parse string into a JsonDocument object / / Get a specific property from JSON
using JsonDocument jd =
JsonDocument.Parse(J sonStri ngs.PERSON); JsonElement je = jd.RootElement
.GetProperty("address") .GetProperty("city");
/ / Get a specific property from JSON
JsonElement je = / / Get the string value from the JsonElement
jd.RootElement!.GetProperty("name");
Console.Wri teLi ne($"Ci ty={j e! .GetStringO}");
/ / Get the numeric value
/ / from the JsonElement After parsing the string into the JsonDocument object, access
Console.Wri teLi ne( the RootElement property and call the GetProperty("address") to
$"Name={je! .GetStri ng()}" ) ; get to the "address" property, and then call GetProperty("city")
Console.Wri teLi ne(
$"Age={jd.RootElement! to get to the "city" property. Once you have this element in
.GetPropertyC'age")! .Getlnt32()}"); a JsonElement object, call the GetStringO method to retrieve
the value for the "city" property.

20 Manipulating JSON Documents in .NET 8 codemag.com

Run the application and you should see the fo llo w in g dis­ Console.Wri teLi ne(
played in the console window: $"City={node! .AsValueO}");

Ci ty=Nashvilie The above code parses the PERSON_ADDRESS string into a

JsonNode object. I t then uses an indexer on the jn variable to
Reading Data Using JsonNode d rill down to the address.city node. This node is placed into
Unlike the JsonDocument object, the JsonNode object a new JsonNode object named node. The value of the "city"
has an indexer th a t allows you to specify the name in property is retrieved using the AsValue() method and dis­
double brackets to retrieve th a t specific node, as shown played on the console window when you run this application.
in the fo llo w in g code:

/ / Parse string into a JsonNode object Add, Edit, and Delete Nodes
JsonNode? jn = To add a new nam e/value pair to a JSON document, cre­
JsonNode.Parse(J sonStrings.PERSON); ate a JsonObject object o u t o f the PERSON JSON string
constant and convert i t to a JsonObject using the AsOb-
/ / Get the age node je c t() method. Once you have a JsonObject, use the Add()
JsonNode? node = jn !["age"]; method to create a new nam e/value pair, in th is case
"hairC olor": "Brown".
W ith th is new JsonNode object, node, retrieve the value
as a JsonValue using the AsValue() method. W ith the using JsonSamples;
JsonValue object, you can report the path o f where th is using System.Text.Json.Nodes;
value came from , the type (strin g , number, Boolean,
etc.), and get the value its e lf as shown in the fo llo w in g / / Parse string into a JsonObject
code: JsonObject? jo = JsonNode.Parse(
JsonStri ngs.PERSON)?.AsObject();
/ / Get the value as a JsonValue
JsonValue value = node!.AsValueO; jo?.Add("hairColor" , JsonValue.Create("Brown"));
Console.WriteLine($"Path={value.GetPath()}");
Console.WriteLine($"Type={value.GetValueKind()}"); Console.WriteLine(jo?.ToString());
Console.WriteLine($"Age={value}");
Replace the code in the Program.es file w ith the code
A nother op tion is to retrieve the value using the listed above and run the application to see the fo llo w in g
GetValue<T>() method, as shown in the fo llo w in g code: displayed in the console window:

/ / Get the value as an integer {

int age = node!.GetValue<int>(); "name": "John Smith",
Console.WriteLine($"Age={age}"); "age": 31,
"ssn": null,
I f you type in the above code snippets in to the Program, "isActive": true,
cs file and run the ap plication, the fo llo w in g should be "hairColor": "Brown"
displayed in the console window: }

Path=$.age Updating a Node

Type=Number Use the JsonNode object to update a value in a name/value
Age=31 pair. Parse the JSON in to a JsonNode object, then access
Age=31 the name using an indexer. Set the value using the equal
sign ju s t as you would any normal assignment in .NET.
Retrieve Data in a Nested Object
Look back a t Listing 1 and look a t the PERSON_ADDRESS using JsonSamples;
constant. This JSON string is the one w ith the nested using System.Text.Json.Nodes;
"address" object. To access the "c ity " value w ith in the
"address", you replace the code in the Program.es file / / Parse string into a JsonNode object
w ith the fo llo w in g : JsonNode? jo = JsonNode.Parse(
JsonStrings.PERSON);
using JsonSamples;
using System.Text.Json.Nodes; jo !["age"] = 42;

/ / Parse string into a JsonNode object Console.WriteLine(jo?.ToString());

JsonNode? jn = JsonNode.Parse(
JsonStrings.PERS0N_ADDRES5); Replace the code in the Program.es file w ith the code
listed above and run the application to see the fo llo w in g
/ / Get the address.city node code displayed in the console window. Notice th a t the
JsonNode? node = age value has changed from 31 to 42.
jn! ["address"]! ["c ity "];
{
/ / Display string value from the JsonNode "name": "John Smith",

codemag.com Manipulating JSON Documents in .NET 8 21

 "age": 42, object separated by commas for each element you wish
"ssn": null, to create in the array. Write the following code into the
"isActive": true Program.es file:
}
using System.Text.Json.Nodes;
Deleting a Node
The Remove() method on a JsonObject is used to delete a JsonArray ja = [
name/value pair from a JSON document. Create a JsonOb­ new JsonObject() {
je c t object out of the JSON PERSON string constant. ["name"] = "John Smith",
Once you have a JsonObject, use the Remove() method, ["age"] = 31
passing in the name you wish to remove from the JSON. }.
In the code below, you remove the "age" name/value new JsonObject() {
pair: ["name"] = "Sally Jones",
["age"] = 33
using JsonSamples; }
using System.Text.Json.Nodes;

// Parse string into a JsonObject Console.WriteLine(ja.ToString());

JsonObject? jo = JsonNode.Parse( Console.WriteLine(ja.GetValueKind());
JsonStrings.PERSON)?.AsObject();
Run the application and you should see the following dis­
jo?.Remove("age"); played in the console window:

Console.WriteLi ne(jo?.ToString());
{
Replace the code in the Program.es file with the code "name": "John Smith",
listed above and run the application to see the following "age": 31
displayed in the console window. The "age": 31 name/ }.
value pair has been removed from the JSON document. {
"name": "Sally Jones",
{ "age": 33
"name": "John Smith", }
"ssn": null, ]
"isActive": true Array
}
Manipulate an Array
Like most arrays in .NET, you can easily add and remove
Working with Arrays elements within the array. Given the previous JsonArray
In addition to a simple object, JSON can contain arrays object declaration, you can insert a new entry into the
of strings, numbers, Booleans, and JSON objects. Instead array by adding the following code after the declaration.
of using the JsonObject to represent a JSON document, The Insert() method lets you specify where in the array
use the JsonArray class to represent a list of items. For you wish to add the new object. In this case, you are
example, to create an array of string values, replace the adding a new element into the first position of the array.
code in the Program.es file with the following:
ja.lnsert(0, new JsonObject() {
using System.Text.Json.Nodes; ["name"] = "Charlie Chaplin",
["age"] = "50"
JsonArray ja = [ "John", "Sally", "Charlie"]; });

Console.Wri teLi ne(j a.ToStri ng()); You can always create a new JsonObject first, initialize it
Console.WriteLine(ja.GetValueKind()); with some data, then add th a t new JsonObject to the Json­
Array using the Add() method. The Add() method adds the
Run the application and you should see the following JsonObject to the end of the array.
code displayed in the console window. Notice th a t after
the JSON array is displayed, the type reported from the JsonObject jo = new() {
call to the GetValueKind() method is "Array". ["name"] = "Buster Keaton",
["age"] = 55
};
"John", j a.Add(jo);
"Sally",
"Charlie" Array elements may be removed by either a reference to
] the actual object, or by using an index number, as shown
Array in the following two lines of code:

To create an array of JSON objects, use the same syntax ja.Remove(jo);

with the square brackets, but create a new JsonObject ja.RemoveAt(2);

22 Manipulating JSON Documents in .NET 8 codemag.com

Extract a JSON Array from a JSON Node Type the above code into the Program.es file and run the
Look back at Listing 1 to view the PERSON_ADDRESS_ application to see the following values displayed in the
PHONES constant string. Within this JSON object, there's console window:
an object named "phoneNumbers" th a t contains an array
of phone number objects. To extract the phone number Type=Home, Phone Number=615.123.4567
objects from this string, you first need to parse the string Type=Mobile, Phone Number=615.345.6789
into a JsonNode object. You then retrieve the value from Type=Work, Phone Number=615.987.6543
"phoneNumbers" object and convert i t into a JsonArray
object using the AsArray() method, as shown in the fo l­ Iterate Over Array Values Using JsonDocument
lowing code: I f you wish to use the JsonDocument class instead of a
JsonNode class, the following code illustrates the differ­
using JsonSamples; ences between the two classes. After parsing the PHONE_
using System.Text.Json.Nodes; NUMBERS string constant into a JsonDocument, convert the
RootElement property, which is an array, into an ArrayEnu-
// Parse string into a JsonNode object merator using the EnumerateArray() method. You may now
JsonNode? jn = JsonNode.Parse( iterate over the array of Json Element objects and display
JsonStri ngs.PERSON_ADDRESS_PHONES); the phone number type and the phone number itself.

// Get the Phone Numbers Array using JsonSamples;

JsonArray? ja = jn !["phoneNumbers"]! .AsArrayO; using System.Text.Json;

Console.WriteLine(ja.ToString()); // Parse string into a JsonDocument object

Console.WriteLine(ja.GetValueKind()); using JsonDocument jd = JsonDocument.Parse(
JsonStrings.PHONE_NUMBERS);
Place this code in to the Program.es file and run the
application to display the follow ing in the console w in­ JsonElement.ArrayEnumerator elements =
dow: jd.RootElement.EnumerateArray();
foreach (JsonElement elem in elements) {
Console.Wri teLi ne(
{ $"Type={elem.GetProperty("type")},
"type": "Home", Phone Number={elem.GetProperty("number")}");
"number": "615.123.4567" }
},
{
"type": "Mobile", Listing 3: Retrieve a single item from an array
"number": "615.345.6789" using JsonSamples;
}, using System.Text.Json.Nodes;
{
string? value = string.Empty;
"type": "Work",
"number": "615.987.6543" // Create JsonNode object from Phone Numbers
} JsonNode? jn = JsonNode.Parse(
JsonStrings.PHONE_NUMBERS);
Array // Cast phone numbers as an array
JsonArray ja = jn! .AsArrayO;
Iterate Over Array Values Using JsonNode // Search for Home number
Once you have a JsonArray object, you may iterate over JsonNode? tmp =
each value in the array to extract the different property ja.FirstOrDefault(row =>
values. In the code shown below, you parse the PHONE_ (string)(row!["type"]!
.GetValue<string>()) == "Home");
NUMBERS string constant into a JsonNode object. Next,
convert this JsonNode object into a JsonArray using the // Extract the home number value
AsArrayO method. Use a foreach loop to iterate over each value = tmp!["number"]! .GetValue<string>();
element in the array and emit the "type" and "number" Console.WriteLine($"Home Number={value}");
properties onto the console window.

using JsonSamples;
using System.Text.Json.Nodes; Listing 4: A sample runtime configuration file

{
// Parse string into a JsonNode object "runtimeoptions": {
JsonNode? jn = JsonNode.Parse( "tfm": "net8.0",
"framework": {
JsonStrings.PHONE_NUMBERS); "name": "Microsoft.NETCore.App",
"version": "8.0.0"
JsonArray? nodes = jn!.AsArrayO; }.
foreach (JsonNode? node in nodes) { "configProperties": {
"System.Runtime.. false
Console.Wri teLi ne($"Type={node!["type" ]}, }
Phone Number={node!["number"]}"); }
} }

codemag.com Manipulating JSON Documents in .NET 8 23

 The output is the same as the output shown above when Parsing JSON from a File
using the JsonNode object to iterate over the array values. There are a couple of different methods you may use to
extract JSON from a file. You can use .NET File I/O classes,
Get a Single Phone Number from Array or you can use the IConfiguration interface. Let's start by
Looking back at Listing 1, you see the PHONE_NUMBERS looking at how you can read JSON values using the .NET
string constant, which is a JSON array. After you parse this File I/O classes.
data into a JsonNode, you might wish to retrieve ju st the
home phone number from this array. After converting the Read Runtime Configuration Settings
phone numbers to a JsonArray object, use the FirstOrDe- When you run a .NET application, there's a runtimecon-
fault() method to search where the "type" value is equal fig.json file (Listing 4) created with information about
to "Home". I f this node is found, extract the number value the application. The JSON object shown below is an ex­
to display on the console window, as shown in Listing 3. ample of what's generated from a console application.
I f you type the code shown in Listing 3 into the Program, When you run an ASP.NET web application, there w ill be
cs file and run the application, the code displays "Home additional information in this file.
Number=615.123.4567" in the console window.
I f you wish to read the .NET Framework version from this
file, you need to first open the file and parse the text into
a JsonNode or JsonDocument object, as shown in Listing
Listing 5: Use File I/O to read a value from the runtime configuration file
5. You then access the runtimeOptions.framework.ver-
using System.Text.Json.Nodes; sion property to retrieve the value "8.0.0". Replace all the
code in the Program.es file with the code shown in Listing
string? value = string.Empty;
string fileName = 5 and run the application to display the runtime version.
$"{AppDomain .CurrentDomai n.Fri endlyName}
. runtimeconfig, j son"; Create an appsettings.json File
i f (File.Exists(fileName)) { In most .NET applications you write, you most likely w ill
JsonNode? jn = JsonNode.Parse( need a file to store global settings such as a connection
File.ReadAllText(fileName)); string, Logging Levels, and other application-specific set­
value = jn ! [ " runtimeoptions"]
! ["framework"] tings. This information is generally stored in a file named
! ["version"]?.GetValue<string>(); appsettings.json. Add this file to the console application
} and put the settings shown in Listing 6 into this file.
Once the file is created, click on the file and bring up the
Console.WriteLine(json);
Properties window. Set the Copy to Output Directory
property to Copy always. You should put the connection
Listing 6: Create an application settings file in the console application string all on one line. I had to break i t across several lines
due to the formatting constraints of the print magazine.
{
"Connectionstrings": {
"Defaultconnection": Read appsettings.json File Using File I/O
"Server=Localhost; Because the appsettings.json file only contains text, you
Database=AdventureWorks;
Trusted_Connection=Yes; can read in the JSON contained in this file using the .NET
MultipleActiveResultSets=true; File I/O classes, as shown in Listing 7. In this code, you
TrustServerCertificate=True;" set the fileName variable to point to the location of the
}. appsettings.json file. Because the appsettings.json file is
"AdvWorksAppSettings": {
"ApplicationName": "Adventure Works", in the same folder as the executable that's running, you
}, don't need to specify a path to the file. I f the file exists,
"Logging": { read all of the text using the ReadAllText() method of
"LogLevel": {
"Default": "Information", the File class and pass th a t text to the Parse() method
"Microsoft.AspNetCore": "Warning" of the JsonNode class. Once you have the JSON in the
} JsonNode object, you can read the value from the Con-
}
} nectionStrings.DefaultConnection property. Type the
code shown in Listing 7 into the Program.es file, run the
application, and you should see the connection string in
Listing 7: Read the appsettings.json file using .NET File I/O classes the appsettings.json file displayed in the console window.

using System.Text.Json.Nodes;
Writing to a File
string connectstring = string.Empty; I f you're running a WPF application, or a console application,
string fileName = "appsettings.json"; it's perfectly acceptable to write data back to the appset­
i f (File.Exists(fileName)) { tings.json file. Of course, you wouldn't want to do this when
// Read settings from file running an ASP.NET web application. The code shown in List­
JsonNode? jd = JsonNode.Parse( ing 8 reads in the appsettings.json file, adds a new name/
File.ReadAllText(fileName)); value pair, then writes the new JSON back to the appsettings.
// Extract the default connection string json file. Type the code in Listing 8 into the Program.es file
connectstring = jd !["Connectionstrings"] and run the application to produce the following results:
! ["Defaultconnection"]?
.GetValue<string>() ?? string.Empty;
} {
"Connectionstrings": {
Console.WriteLine(connectString); "Defaultconnection": "Servers..;"

24 Manipulating JSON Documents in .NET 8 codemag.com

 }. Listing 8: Write a new value to the appsettings.json file
"AdvWorksAppSettings": {
"AppticationName": "Adventure Works", string fileName = "appsettings.json";
JsonObject? jo = null;
"LastDateUsed": "12/27/2023"
}. i f (File.Exists(fileName)) {
"Logging": { // Read settings from file
jo = JsonNode.Parse(
"LogLevel": { File.ReadAllText(fileName))?.AsObj ect();
"Default": "Information",
"Microsoft.AspNetCore": "Warning" i f (jo != null) {
/ / Locate node to add to
} JsonObject? node =
} jo ! [ "AdvWorksAppSetti ngs"]?.AsObject();
}
/ / Add new node
node?.Add("LastDateUsed",
JsonValue.Create(
Using the IConfiguration Interface DateTi me.Now.ToShortDateStri ng()));
Instead of using the .NET File I/O classes, you can take
/ / Write back to file
advantage of the IConfiguration interface and the Con­ File.WriteAHText(fileName, jo?.ToString());
figuration Builder class to read in a JSON file. To use the }
Configuration Bui Ider class, you must add two packages to }
your project. Console.WriteLi ne(jo?.ToString());

• Microsoft. Extensions.Configuration
• Microsoft. Extensions.Configuration. Json Listing 9: Use the Configuration Builder class to read in a JSON file

After adding these two packages to your project, you can using Microsoft.Extensions.Configuration;
write the code shown in Listing 9. In this code, you pass string? value = string.Empty;
in the runtime configuration file name (see Listing 4) string fileName =
to the AddJsonFile() method on the Configuration Builder. $"{AppDomai n.CurrentDomain.FriendlyName}
.runtimeconfig.json";
The Build() method is called to create the configuration
builder object, which reads the JSON file into memory IConfiguration config =
and converts the text into a JSON document. Use the new ConfigurationBuiIder()
.AddJsonFile(fileName)
GetSection() method to retrieve a specific section within .BuildO;
the JSON file. In this case, you're asking for the runtime-
Options section. From the section variable, you can now IConfigurationSection section =
config.GetSection("runtimeoptions");
retrieve the framework version number. Type in the code value = section["framework:version"]
in Listing 9 into the Program.es file, run the applica­ ?? string.Empty;
tion, and you should see the version number appear in
Console.Wri teLi ne(value);
the console window.

Read the appsettings.json File

You previously read the appsettings.json file using the Listing 10: Read the appsettings.json file using the Configuration Builder class
.NET File I/O classes. In Listing 10 you're now going to using Microsoft.Extensions.Configuration;
use the Configuration Builder to read the same file. Be­
cause the appsettings.json file is in the same folder as string connectstring;
the executable that's running, you don't need to specify IConfiguration config =
a path to the file. Type the code in Listing 10 into the new ConfigurationBuiIder()
Program.es file, run the application and you should see .AddJ sonFile("appsetti ngs.j son")
the connection string appear in the console window. .BuildO;

IConfigurationSection section =
Bind Settings to a Class config.GetSection("Connectionstrings");
Instead of reading values one at a time from a configura­ connectstring =
section["Defaultconnection"]
tion file, you can bind a section within a configuration ?? string.Empty;
file to a class with ju s t one line of code. Create a class
named AppSettings and add a property th a t maps to each Console.WriteLine(connectstring);
name in the configuration file. In the following code,
there's a sole property named Application Name that
maps to the "ApplicationName" property in the appset­ To perform the binding operation, add the package Micro-
tings.json file shown in Listing 6. soft.Extensions.Configuration.Binder to your project
using the NuGet Package Manager. Add the following code
namespace JsonSamples; to the Program.es file and run the application to see the
application name displayed in the console window:
public class AppSettings
{ using JsonSamples;
public string ApplicationName { get; set; } using Microsoft.Extensions.Configuration;
= string.Empty;
} AppSettings entity = new();

codemag.com Manipulating JSON Documents in .NET 8 25

 public class Person
iconfiguration config = {
new ConfigurationBuiIder() public string? Name { get; set; }
.AddJsonFile ("appsettings.j son") public int Age { get; set; }
,Build(); public string? SSN { get; set; }
public bool IsActive { get; set; }
config.Bi nd("AdvWorksAppSettings", entity);
public override string ToStringO
Console.Wri teLi ne(enti ty.Appli cati onName); {
return $"{Name}, Age={Age},
SSN={SSN}, IsActive={IsActive}";
Serialize C# Object to JSON }
So far, everything you've done manipulates JSON using C# }
code. Another excellent feature o f .NET is th a t you may se­
rialize your C# objects in to JSON using ju s t a few lines of A fte r creating the Person class, replace a ll the code in the
code. This is very handy for sending C# objects over the Program.es file w ith the fo llo w in g :
internet via Web API calls. In fact, the code you're going
to Learn about now is exactly how ASP.NET sends your data using JsonSamples;
across the internet when w riting Web API calls. To illustrate using System.Text.Json;
this concept, rig ht mouse-click on the project and add a new
class named Person, as shown in the follow ing code snippet: Person entity = new() {
Name = "John Smith",
namespace JsonSamples; Age = 31,
SSN = null,
IsActive = true
};
Listing 11: Add a JsonSerializerOptions ob je ct to co n tro l how the JSON is form atted

using JsonSamples; Console.Wri teLi ne(

using System.Text.Json; JsonSeri ali zer.Seri ali ze(enti ty));

Person entity = new() { This code uses the Serialize() method of the JsonSerializer
Name = "John Smith",
Age = 31, class from the System.Text.Json namespace. Pass in the in ­
SSN = null, stance o f your C# object to the Serialize() method and a string
IsActive = true of JSON is returned. Run the application and you should see
};
the following string of JSON appear in your console window:
JsonSerializerOptions options = new() {
PropertyNamingPolicy = {"Name":"John Smith","Age":31,
JsonNamingPolicy.CamelCase,
Writelndented = true "SSN"m ull,"IsActive":true}
};
Notice th a t there's no in d e n ta tio n or spaces between the
Console.Wri teLi ne(
JsonSerializer.Serialize(entity, options)); values. Also notice th a t the names are the exact same
case as your C# class property names. JSON usually uses
camel case fo r names (firs t le tte r is Lower-case), whereas
C# uses Pascal case (firs t Letter is upper-case).
Listing 12: Add JSON a ttrib u te s to your C# class properties to con trol serialization

using System.Text.Json.Serialization; Change Casing o f Property Names

I f you wish to change the casing o f the property names,
namespace JsonSamples;
create an instance o f a JsonSerializerOptions ob je ct and
public class PersonWithAttributes set the PropertyNamingPolicy to the enum eration value
{ o f CamelCase (as seen in Listing 1 1 ). Change the fo r­
[JsonPropertyName("personName")]
[JsonPropertyOrder(1)] m attin g o f the JSON to become indented by se ttin g the
public string? Name { get; set; } W ritelnd en ted property to true.
[JsonPropertyName("personAge")]
[JsonPropertyOrder (2)]
public int Age { get; set; } Type the code in Listing 11 in to the Program.es file and
public string? SSN { get; set; } run the application to display the fo llo w in g JSON in the
public bool IsActive { get; set; } console window:

[Jsonlgnore]
public DateTime? CreateDate { get; set; }
[Jsonlgnore(Condition = "name": "John Smith",
JsonlgnoreCondition.WhenWritingNull)] "age": 31,
public DateTime? ModifiedDate { get; set; }
"ssn": null,
public override string ToStringO "IsActive": true
{
return $"{Name}, Age={Age},
SSN={SSN}, IsActive={IsActive}";
} Go back to the Program.es file and change the Proper­
} tyNamingPolicy property to JsonNamingPolicy.Snake-

26 Manipulating JSON Documents in .NET 8 codemag.com

CaseUpper to produce each property as upper-case w ith Listing 13: Serialize the C# object w ith the JSON a ttrib u te s applied
each word in the property name separated by an under­
score, as shown in the fo llo w in g o u tp u t:
using JsonSamples;
using System.Text.Json;

{ PersonWithAttributes entity = new() {

Name = "John Smith",
"NAME": "John Smith", Age = 31,
"AGE": 31, SSN = null,
"SSN": null, IsActive = true,
// This property is never serialized
"IS_ACTIVE": true CreateDate = DateTime.Now,
} // Comment this property to
// remove from serialization
ModifiedDate = DateTime.Now
Go back to the Program.es file and change the Proper­
};
tyNamingPolicy property to JsonNamingPolicy.Snake-
CaseLower to produce each property as Lower-case w ith JsonSerializerOptions options = new() {
PropertyNamingPolicy =
each word in the property name separated by an under­ JsonNamingPolicy.CamelCase,
score, as shown in the fo llo w in g o u tp u t: Writelndented = true
};
{ Console.Wri teLi ne(
"name": "John Smith", JsonSerializer.Serialize(entity, options));
"age": 31,
"ssn": null,
"is_active": true {
Getting the Sample Code
} "ssn": null,
You can download the sample
"isActive": true,
code for this article by visiting
The oth er enum eration values you may set the Proper­ "modifiedDate": "2024-01-28T12:48:53", www.CODEMag.com under the
tyNam ingPolicy to are JsonNamingPolicy.KebabCaseUp- "personName": "John Smith", issue and article, or by visiting
per or JsonNamingPolicy.KebabCaseLower. This policy "personAge": 31 www.pdsa.com/downloads.
separates each word in the property name w ith a dash } Select "Articles" from the
instead o f an underscore, as shown below: Category drop-down. Then
There are a few th in g s to notice about th is o u tp u t. The select "XML Serialization and
{ m odifiedDate appears in the o u tp u t because the value is Validation in .NET6/7"from
"name": "John Smith", n o t null. The Name and Age C# properties are em itte d as the Item drop-down.
"age": 31, personName and personAge in JSON. These tw o proper­
"ssn": null, ties also appear a t the end o f the ob je ct because th e ir
"is-active": true order was set to one (1) and tw o (2) respectively. I f you
} go back to the code in the Program.es file , comment the
M odifiedDate property, and rerun the application, the
ontrol Serialization Using JSONAttributes M odifiedDate value w ill n o t display in the ou tp u t.
In the System .Text.Json.Serialization namespace are
some a ttrib u te s you may use to decorate C# class proper­ Serializing Objects with Enumerations
ties to help you co n tro l how each property is serialized. A nother common scenario is th a t you have a class w ith an
There are several a ttrib u te s you may use, b u t the most enum eration as one o f the property types. When you se­
used are Json Property Name, Jsonlgnore, and JsonProp- rialize th a t object, you can e ith e r e m it the numeric value
ertyOrder. Listing 12 shows a Person W ith A ttn butes o f the enum eration or use the string representation o f
class w ith these a ttrib u te s applied to d iffe re n t properties. the enum eration its e lf. Create a new enum eration named
PersonTypeEnum.
When you decorate a C# property w ith the JsonProper-
tyNam e a ttrib u te , you pass the JSON name to use to the namespace JsonSamples;
a ttrib u te when th is property is serialized. When a C# class
is serialized, the order o f the properties is in the order public enum PersonTypeEnum
they appear in the class. To change the order in which {
the properties are serialized, add a JsonPropertyOrder Employee = 1,
a ttrib u te to each property and set the order in which you Customer = 2,
w ant them to appear. I f the JsonPropertyOrder a ttrib u te Supervisor = 3
is no t applied to a property, the default number is zero }
(0 ). To never have a property serialized in to JSON, apply
the Jsonlgnore a ttrib u te w ith no parameters. You may Create a class named PersonWithEnum th a t has a Per-
also set the Condition property o f the Jsonlgnore a t­ sonType property th a t is o f the type PersonTypeEnum.
trib u te to n o t serialize the data when the value is null.
namespace JsonSamples;
A fte r creating the Person W ith A ttn butes class, w rite the
code shown in Listing 13 in the Program.es file . In th is public class PersonWithEnum
lis tin g , notice th a t the CreateDate and M odifiedDate {
properties are both set to the current date and tim e. I f public string? Name { get; set; }
you run the code shown in Listing 13, the fo llo w in g o u t­ public PersonTypeEnum PersonType { get; set; }
p u t is displayed in the console window:

codemag.com Manipulating JSON Documents in .NET 8

 Listing 14: Create a Jw tS ettings class to be nested w ith in an AppSettings class

namespace JsonSamples; public string[] AllowedlPAddresses

{ get; set; } = [];
public class JwtSettings
{ #region ToString Override
public string Key { get; set; } public override string ToStringO
= string.Empty; {
public string Issuer { get; set; } return $"{Key} - {Issuer} -
= string.Empty; {Audience} - {MinutesToExpirati on}" ;
public string Audience { get; set; } }
= string.Empty; #endregion
public int MinutesToExpirati on }
{ get; set; }

Listing 15: W rite code to serialize a nested ob je ct to view the JSON o u tp u t

using JsonSamples;
using System.Text.Json;
AppSettingsNested entity = new() { JsonSerializerOptions options = new() {
ApplicationName = "JSON Samples", PropertyNamingRolicy = JsonNamingPolicy.CamelCase,
JWTSettings = new() { Writelndented = true
Key = "ALongKeyForASymmetricAlgorithm", };
Issuer = "JsonSamplesAPI",
Audience = "PDSCJsonSamples", Console.Wri teLi ne(
MinutesToExpi rati on = 60 JsonSerializer.Serialize(entity, options));

public override string ToStringO er class. This class works w ith the serializer and instead
{ o f e m ittin g the numeric value o f enum eration properties,
return $"{Name}, Type={PersonType}"; i t em its the string representation o f the enum eration.
}
} JsonSerializerOptions options = new() {
PropertyNamingPolicy =
Open the Program.es file and w rite the code shown in the JsonNamingPolicy.CamelCase,
code snippet below: Writelndented = true,
Converters =
using JsonSamples; {
using System.Text.Json; new JsonStringEnumConverter()
}
PersonWithEnum entity = new() { };
Name = "John Smith",
PersonType = PersonTypeEnum.Supervisor A fte r adding the JsonStringEnumConverter object, run
}; the application and the fo llo w in g should now display in
the console window:
JsonSerializerOptions options = new() {
PropertyNamingPolicy = JsonNamingPolicy.CamelCase, {
Writelndented = true "name": "John Smith",
}; "personType": "Supervisor"
}
Console.Wri teLi ne(
JsonSerializer.Serialize(entity, options)); Serialize a Nested Object
Often you have a property in a class th a t is its e lf another
When you run the ap plication, the fo llo w in g o u tp u t is class. Don't worry, the JSON serialization process handles
displayed in the console window. this situation ju s t fine. To illustrate, create a class called
JwtSettings, as shown in Listing 14. Next, create a class
{ named AppSettingsNested th a t has tw o properties: Appli-
"name": "John Smith", cationName and JWTSettings. The data type for the JWT-
"personType": 3 Settings property is the JwtSettings class you ju s t created.
}
namespace JsonSamples;
Notice th a t the personType property has a value o f 3,
which equates to the Supervisor enum eration value. Go public class AppSettingsNested
back to the Program.es file and add the fo llo w in g using {
statem ent a t the to p o f the file : public string ApplicationName
{ get; set; } = string.Empty;
using System.Text.Json.Serialization;
public JwtSettings JWTSettings
Set the Converters property in the JsonSerializerOptions { get; set; } = new();
ob je ct to use an instance o f the JsonStringEnumConvert- }

28 Manipulating JSON Documents in .NET 8 codemag.com

Now th a t you have a nested class, write the code shown in Listing 16: Create a generic List<T> and serialize i t to create a JSON array
Listing 15 into the Program.es file. In this code, you f ill in
the Application Name property, create a new instance of a
using JsonSamples;
using System.Text.Json;
JwtSettings class for the JWTSettings property, and then f ill
in each property in th a t class too. Run the application and you List<PersonWithEnum> lis t = new()
should see the following displayed in your console window: {
new PersonWithEnum {
Name = "John Smith",
{ PersonType = PersonTypeEnum.Supervisor
"applicationName": "JSON Samples", },
new PersonWithEnum {
"jwtSettings": { Name = "Sally Jones",
"key": "ALongKeyForASymmetrieAlgorithm", PersonType = PersonTypeEnum.Employee
"issuer": "JsonSamplesAPI", }
};
"audience": "PDSCJsonSamples",
"minutesToExpiration": 60, JsonSerializerOptions options = new() {
PropertyNamingPolicy =
"allowedIPAddresses": [] JsonNamingPolicy.CamelCase,
} Writelndented = true
} };
Console.Wri teLi ne(
Serialize a Dictionary JsonSerializer.Seri alize(list, options));
I f you have a generic Dictionary<TKey, TValue> object, or
a KeyValuePair ob je ct fille d w ith data, the JSON serializer
em its those as a single object. Open the Program.es file Listing 17: Use the Deserialize() method to convert a JSON string in to a C# object
and add the code shown below.
using JsonSamples;
using System.Text.Json;
using System.Text.Json;
// JSON names must match
// C# properties by default
Dictionary<string, object?> diet = new() string json = @"{
{ ""Name"": ""John Smith"",
{"name", "John Smith"}, ""Age"": 31,
""SSN"": null,
{"age", 31}, ""IsActive"": true
{"isActive", true} }";
}; // Deserialize JSON string into Person
Person? entity = JsonSerializer
JsonSerializerOptions options = new() { .Deserialize<Person>(json);
Writelndented = true Console.WriteLine(entity);
};

Console.Wri teLi ne(

Listing 18: Pass in options to co n tro l the deserialization process
JsonSerializer.Serialize(dict, options));
using JsonSamples;
When you run the ap plication, the console window dis­ using System.Text.Json;
plays the fo llo w in g JSON object: string json = @"{
""name"": ""John Smith"",
""age"": 31,
{ ""ssn"": null,
"name": "John Smith", ""isActive"": true
"age": 31, }";
"isActive": true // Override case matching
} JsonSerializerOptions options = new() {
PropertyNameCaselnsensitive = true
Serialize a List };
To w rite a JSON array, you can use any o f the lEnumerable // Deserialize JSON string into Person
objects in .NET such as an array or a List<T>. To illu stra te , Person? entity =
JsonSerializer.Deserialize<Person>(json, options);
use the Person W ith Enum class and create a generic lis t
o f tw o PersonWithEnum objects, as shown in Listing 16. Console.WriteLine(entity);
Type the code shown in Listing 16 in to the Program,
cs file and run the application to display the fo llo w in g
o u tp u t in the console window. }
]

{
"name": "John Smith", Deserialize JSON into a C# Object
"personType": 3 Now th a t you've seen how to serialize a C# object in to
}. JSON, let's Look at reversing the process. To illustrate, cre­
{ ate a JSON string w ith the JSON property name th a t exactly
"name": "Sally Jones", matches the C# property name in the class you wish to dese­
"personType": 1 rialize this JSON in to . You use the JsonSerializer class (List-

codemag.com Manipulating JSON Documents in .NET 8 29

 Listing 19: Use the JsonNode ob je ct to deserialize a JSON string in to a C# object

using JsonSamples; PropertyNameCaselnsensitive = true,

using System.Text.Json.Nodes; };
using System.Text.Json;
// Parse string into a JsonNode object
string json = @"{ JsonNode? jn = JsonNode.Parse(json);
""name"": ""John Smith"",
""age"": 31, // Deserialize JSON string into Person
""ssn"": null, // using the JsonNode object
""isActive"": true Person? entity =
jn.Deserialize<Person>(options);

JsonSerializerOptions options = new() { Console.WriteLine(entity);

ing 17) Like you did for serializing, but call the Deserialize() case. I f you fo rg e t to use the options and the property
method passing in the data type you wish to deserialize the names have d iffe re n t casing, then no data is mapped from
JSON string in to and the string itself. Type the code in List­ the JSON to the C# object, so an em pty ob je ct is returned
ing 17 in to the Program.es file and run the application to from the Deserialize() method.
see the follow ing results displayed in the console window:
Deserialize Using the JsonNode O bject
John Smith, Age=31, SSN=, IsActive=True A nother op tion fo r deserializing JSON in to a C# ob je ct is
to use e ith e r the JsonNode or the JsonDocument classes.
Use S erialization O ptions The code shown in Listing 19 uses the JsonNode object
Just like you did when serializing, i f the case o f the JSON to illu s tra te . The JsonDocument class looks very sim ilar to
property names doesn't match the C# property names, you th a t o f the JsonNode. Type th is code in to the Program,
may set the PropertyNameCaselnsensitive property to cs file and run the application to get the same o u tp u t as
true in the options and pass those options to the Dese- you saw in the Last example.
ria lize () method, as shown in Listing 18. Notice th a t in
th is lis tin g the JSON property names s ta rt w ith Lower- Deserializing E num eration Values
I f you know th a t the JSON object is going to have the
string representation o f a C# enum eration, set the Con­
verters property to a new instance o f the JsonString-
EnumConverter class in the JsonSerializerOptions object,
ADVERTISERS INDEX as shown in Listing 20. I f you forg et to include the Con­
verters property on the JsonSerializerOptions, a Json Ex­
ception is thro w n. Type the code shown in Listing 20
Advertisers Index in to the Program.es file and run the application to see
the fo llo w in g o u tp u t appear in the console window:
CODE Consulting--AI Services
www.codemag.com/ai-services 2 John Smith, Type=Supervisor

CODE Consulting C onvert JSON File to a Person O bject

www.codemag.com/Code 75 Right mouse-click on the JsonSamples console applica­
tio n and create a new fold er named JsonSampleFiles.
CODE Staffing
Right m ouse-click on the JsonSampleFiles fold er and add
w w w .codem ag.com /staffing 76
a new file named person.json. Place the fo llo w in g JSON
dtSearch object in to th is file :
www.dtSearch.com 13

Devlntersection {
"name": "Sally Jones",
www.devintersection.com 15
Advertising Sales: "age": 39,
Tammy Ferguson LEAD Technologies "ssn": "555-55-5555",
832-717-4445 ext 26
www.leadtools.com 5 "isActive": true
[email protected]
}

Open the Program.es file and replace the contents o f the

file w ith the code shown in Listing 21. In th is sample,
you're using a FileStream ob je ct to open the file and
stream i t to the Deserialize() method. Thus, you should
use the Using statem ent in fro n t o f the FileStream decla­
ration so i t w ill be disposed o f properly. Run the applica­
tio n and you should see the fo llo w in g o u tp u t displayed in
This listing is provided as a courtesy
to our readers and advertisers. the console window:
The publisher assumes no responsi­
b ility for errors or omissions. Sally Jones, Age=39,
SSN=555-55-5555, IsActive=True

30 Manipulating JSON Documents in .NET 8 codemag.com

 Listing 20: I f you use enumerations, be sure to include the JsonStri ng Enum Converter in the serialization options

using JsonSamples; / / thrown on DeserializeO

using System.Text.Json.Serialization; Converters =
using System.Text.Json; {
new JsonStringEnumConverter()
string json = @"{ }
""name"": ""John Smith"", };
""personType"": ""Supervisor""
// Deserialize JSON string
PersonWithEnum? entity = JsonSerializer
JsonSerializerOptions options = new() { .Deserialize<PersonWithEnum>(json, options);
PropertyNameCaselnsensitive = true,
// Avoid an exception being Console.WriteLine(entity);

Convert a JSON A rray in a File to a List o f Person Objects Listing 21: Read a JSON file and convert the JSON object in the file in to a C# object
Right mouse-click on the JsonSampleFiles fold er and add using JsonSamples;
a new file named persons.json. Place the fo llo w in g JSON using System.Text.Json;
array in to th is file :
string fileName =
$"{AppDomain.CurrentDomain.BaseDi rectory}
JsonSampleFi lesWperson. json";
{
"name": "John Smith", using FileStream stream = File.OpenRead(fileName);
"age": 31,
JsonSerializerOptions options = new() {
"ssn": null,
PropertyNameCaselnsensitive = true,
"isActive": true };
}.
{ // Deserialize JSON string into Person
Person? entity = JsonSerializer
"name": "Sally Jones",
.Deserialize<Person>(stream, options);
"age": 39,
"ssn": "555-55-5555", Console.WriteLine(entity);
"isActive": true
}
Listing 22: Read an array o f JSON objects from a file and convert to a List o f person objects

Open the Program.es file and type in the code shown in using JsonSamples;
using System.Text.Json;
Listing 22. This code is almost the same as the code you
wrote to deserialize a single person object; the only d iffe r­ string fileName =
ence is th a t you pass the data type List<Person> to the De­ $"{AppDomain.CurrentDomain.BaseDi rectory}
serializeO method. Once you have the collection o f Person JsonSampleFi lesWpersons. json";
objects, iterate over the collection and display each person
using FileStream stream = File.OpenRead(fileName);
on the console window. Run the application and you should
see the follow ing o u tp u t displayed in the console window: JsonSerializerOptions options = new() {
PropertyNameCaselnsensitive = true,
John Smith, Age=31, };
SSN=, IsActive=True // Deserialize JSON string into List<Person>
Sally Jones, Age=39, List<Person>? lis t = JsonSerializer
5SN=555-55-5555, IsActive=True .Deserial1ze<List<Person»(stream, options);

i f (lis t != null) {
Get M a x im u m Age from List o f Person Objects foreach (var item in lis t) {
A fte r reading in a lis t o f objects, you may now use LINQ Console.WriteLine(item);
operations or any Enumerable methods such as Min, Max, }
Sum, and Average on th a t List. In the code shown in List­ }
ing 23, the Max() method is applied to the List and the
maximum value found in the Age property is displayed on
the console window. When you run th is ap plication, the able interface, so you need to prefix them w ith the Using
value reported back should be th irty -n in e (39). statem ent. You must s ta rt each JSON docum ent by calling
the W riteS tartO bject() or the W riteStartArray() method.
You then call the appropriate method to w rite a string , a
Using the Utf8JsonWriter Class number, a Boolean, a null, or a comment. Finally, call the
The U tf8JsonW riter class is a high-perform ance, forward- W riteEndObject() or the W riteEndArray() method to close
only, non-cached method o f w ritin g JSON documents. the JSON document. Type the code shown in Listing 24
Just Like w ith serialization, you can co n tro l the o u tp u t o f in to the Program.es file and run the application to display
the JSON to include w h ite space, and in d e n ta tio n . List­ the o u tp u t shown below in the console window:
ing 24 shows how to w rite a single JSON ob je ct in to a
Memorystream object. Note th a t both the Memorystream {
and the U tf8JsonW riter objects im plem ent the IDispos- "name": "John Smith",

codemag.com Manipulating JSON Documents in .NET 8 31

 Listing 23: A fte r deserializing a List o f person objects, apply the Max() method to calculate the largest numeric value

using JsonSamples;
using System.Text.Json;
/ / Deserialize JSON string
string fileName = List<Person>? lis t = JsonSerializer
$"{AppDomain.CurrentDomai n.BaseDi rectory} .Deserialize<List<Person>>(stream, options);
JsonSampleFi lesWpersons. json";
/ / Calculate maximum age
using FileStream stream = File.OpenRead(fileName); int maxAge = lis t? .Max(row => row.Age) ?? 0;

JsonSerializerOptions options = new() { Console.Wri teLi ne(maxAge);

PropertyNameCaselnsensitive = true,

Listing 24: The Utf8JsonW nter object is a forward-only cursor fo r e m ittin g JSON quickly

using System.Text.Json; writer.Wri teStartObj ect();

using System.Text; writer.WriteString("name", "John Smith");
writer.WriteNumber("age", 31);
JsonWriterOptions options = new() { writer.WriteBoolean("isActive", true);
Indented = true writer.Wri teEndObject();
writer.Flush();

using Memorystream ms = new(); string json = Encoding.UTF8

using Utf8JsonWriter writer = .GetString(ms.ToArrayO);
new(ms, options);
Console.Wri teLi ne(j son);

Listing 25: The Utf8JsonW riter o b je ct can w rite arrays as w ell as single objects

using System.Text.Json; writer.WriteBoolean("isActive", true);

using System.Text; writer.Wri teEndObject();
writer.Wri teStartObj ect ();
JsonWriterOptions options = new() { writer.WriteString("name", "Sally Jones")
Indented = true writer.WriteNumber("age", 39);
}; writer.WriteBoolean("isActive", true);
writer.Wri teEndObject();
using Memorystream ms = new(); writer.WriteEndArrayO;
using Utf8JsonWriter writer = writer.Flush ();
new(ms, options);
string json = Encoding.UTF8
writer.Wri teStartArray(); .GetString(ms.ToArrayO);
writer.Wri teStartObj ect();
writer.WriteString("name", "John Smith"); Console.Wri teLi ne(j son);
writer.WriteNumber("age", 31);

SPONSORED SIDEBAR "age": 31, "age": 39,

"isActive": true "isActive": true
CODE Is Hiring!
CODE Staffing is accepting Write a JSONArray
resumes for various open As ju s t mentioned, you may also w rite a JSON array o f data
positions ranging from using the Utf8JsonW riter class. The only difference is th a t Sum m ary
junior to senior roles.
you start w ritin g using the W riteStartArray() method, and In th is article, you were introduced to the many d iffe re n t
then call the W riteStartObject() method to create your firs t classes in .NET th a t are used to m anipulate JSON docu­
We have multiple
openings and will consider JSON object in the array. You then continue th is process ments. I f you need fast, read-only access to JSON docu­
candidates who seek u n til a ll your array elements are w ritte n . Finish the JSON ments, the JsonDocument class is w hat you should use. I f
full-time employment or array document by calling the WriteEndArrayO method, as you need the a b ility to m odify JSON in your application,
contracting opportunities. shown in Listing 25. Type the code shown in Listing 25 use the JsonNode class. Serialization is accomplished us­
For more information: in to the Program.es file and run the application to display ing the JsonSerialization, JsonDocument, or the Json­
www.codestaffing.com. the o u tp u t shown below in the console window: Node classes. When you're dealing w ith configuration
file s such as the appsettings.json file in your application,
take advantage o f the IC onfiguration interface and the
{ Configuration Builder class. Finally, to w rite JSON docu­
"name": "John Smith", ments one nam e/value pair a t a tim e, the U tf8JsonW riter
"age": 31, class gives you the most co n tro l over how your docum ent
"isActive": true is form atted.
}.
{ Paul D. S heriff
"name": "Sally Jones",

32 Manipulating JSON Documents in .NET 8 codemag.com

 ONLINE QUICK ID 2405041

Value Object's New Mapping:

EF Core 8 ComplexProperty
EF Core 8 was released late in 2023 and, if you haven't kept up, there are some important and interesting things to be aware of
There were over 100 tweaks and additions and another 125 bug fixes. I'll be highlighting those that are most important and a
few that piqued my interest or just my curiosity. If you want to explore all of the enhancements and new features on GitHub,

here's a lin k fo r those issues: https://bit.ly/E FC ore8Fea- have a firs t name property and a Last name property in a
tures. The lis t o f issues fo r the fixes can be perused at Customer class.
https://bit.Ly/EFC ore8Fixes.
public class Customer
I f you've follow ed my tech wanderings over the years, {
i t may be no surprise th a t my A#1 favo rite new feature public int Customerld { get; set; }
is the ComplexProperty mapping, an alternative to using public string FirstName { get; set; }
Owned Entities to map complex types and value objects. public string LastName { get; set; }
The new ComplexProperty mapping provides a far superior public DateOnly FirstPurchase { get; set; }
way to map complex types (and therefore, value objects) } Julie Lerman
than the Owned E ntity mapping we've been using since @julielerman
the beginning o f EF Core. To be clear, there are s till some Instead o f using tw o string types fo r every class th a t needs
th edatafarm .com/co nta ct
scenarios th a t are n o t y e t supported, so you may end up a person's name, you can create a new class th a t only has
those tw o strings. Julie Lerman is a Microsoft
using a m ix o f the tw o mappings u n til ComplexProperty is
Regional director, Docker
complete. It's the team's in te n tio n fo r th is to eventually
public class Customer Captain, and a long-time
replace Owned Entities in th e ir entirety. ComplexProperty
{ Microsoft MVP who now
is a big deal and i t was a big deal fo r the team to execute.
public int Customerld { get; set; } counts her years as a coder
It's a t the to p o f th e ir "w hat's new" lists as w ell.
public PersonName Name { get; set; } in decades. She makes
public DateOnly FirstPurchase { get; set; } her living as a coach and
A lthough the OwnsOne and OwnsMany mappings have consultant to software
fu lfille d the basic need to map classes th a t are used as
}
public class PersonName teams around the world.
properties o f e n titie s, the work th a t they were doing un­ You can find Julie present­
{
der the covers was com plicated and led to numerous side ing on Entity Framework,
public string FirstName { get; set; }
effects. The team had tweaked the in ne r Logic a number Domain-Driven Design and
public string LastName { get; set; }
o f tim es across versions, creating breaking changes along other topics at user groups
}
the way, b u t never really solved the problem properly. and conferences around
They have been contem plating a replacement fo r some the world. Julie blogs at
PersonName is a complex type and can be used as a prop­
tim e and have fin a lly pulled i t off. thedatafarm.com/blog, is
erty o f any other class, such as th is Ship Label class, which
the author of the highly
is obviously missing an address b u t th a t's only to keep
There are some caveats, however, which are a few capabili­ acclaimed "Programming
th is explanation simple.
ties th a t didn't make i t in to EF Core 8 bu t w ill be ready for Entity Framework" books,
EF Core 9. In those cases, we ju s t continue using the owned and many popular videos
public class ShipLabel
on Pluralsight.com.
e n tity mappings. I 'll explain the caveats after I allow you to {
feast your eyes on the new ComplexProperty mapping. public int Id { get; set; }
pubic DateOnly Printed {get; set;}
public PersonName Name { get; set; }
TLDR Complex Types and Value Objects }
Let's be sure we're a ll on the same page. A complex type
is a class th a t doesn't have any id e n tity and is used as The most im p o rta n t a ttrib u te o f PersonName is th a t i t
a property o f another class. An easy example is i f you has no id e n tity .

Listing 1: PersonName class im plem ented as a value object

public class PersonName {

{ return obj is PersonName name &&
public PersonName(string firstName, FirstName == name.FirstName &&
string LastName) LastName == name.LastName;
{ }
FirstName = firstName;
LastName = LastName; public override int GetHashCode()
} {
public string FirstName { get; in it; } return HashCode.Combine(FirstName,
public string LastName { get; in it; } LastName);
}
public override bool Equals(object? obj) }

codemag.com Value Object's New Mapping: EF Core 8 ComplexProperty

 A value object is a critical Domain-Driven Design con­ Hello, or Welcome Back, Complex
struct th a t enhances a complex type by ensuring th a t it s
immutable and th a t its equality is always based on the Properties
values of every property in the type by overriding the Entity Framework, and I mean pre-EF Core, had a concept
Equals and GetHashCode methods. Listing 1 shows a ver­ of complex property mappings th a t were more natural
sion of Person Name that's defined as a value object. than owned entities. EF Core 8 harkens back to th a t con­
cept, although with a different implementation. EF Core
Whether PersonName is a simple complex type or a value s till won't make an assumption th a t a complex type is
object, EF Core w ill see that type in its model when it's anything but a malformed entity th a t needs a key prop­
used as a property of another entity, such as Customer. erty. But we have a new way to map i t with the Complex-
However, i t w ill balk because it can only assume that it's Property mapping.
another entity but can't figure out what to do with i t be­
cause i t has no key property. This results in a runtime ex­ Whereas previously you'd have used the OwnsOne method
ception when EF Core is trying to work out the data model. for the owned property, now you use the ComplexProperty
method.

The Problem (s) with Owned Entities override protected void

Owned entities originally came into EF Core to enable this OnModelCreating(ModelBuiIder modelBuiIder)
mapping. They were a new paradigm for handling com­ {
plex types, different from EF6 and earlier. By mapping //modelBuiIder.Enti ty<Customer>()
the Name property of Customer as an owned entity (using .0wns0ne(c => c.Name);
the OwnsOne mapping), EF Core knows th a t it's okay that modelBuiIder.Entity<Customer>()
there's no key. However, in order to track and persist it, .ComplexProperty(c => c.Name);
EF Core, under the covers, treats th a t PersonName object }
as an entity in a relationship with Customer. I t does so
by using a shadow property to infer a key in memory but Like OwnsOne, ComplexProperty has its own methods
ensures th a t the values are stored as individual fields in where you can further define the property, for example,
the Customers table in the database. That is the default tying i t to a backing field or specifying th a t it's required.
behavior. You can configure the mapping to store the val­
ues in a separate database table. But what's most important is th a t EF Core ju s t treats this
as a property and when storing it, explodes the FirstName
I t was a very clever solution that leveraged the existing and LastName properties out to fields in the Customer's
behavior of EF Core. However, because of the complexity of table (by default). I t doesn't set up a fake relationship or
faking the key, there were problematic side effects. For ex­ fake key and then have to tangle with those every time
ample, EF Core couldn't comprehend if you Left the property you track, save, or retrieve data. Because it's not being
null or needed to edit it. Some of those problems were re­ treated as a separate entity, you can also share i t among
solved but there are others still. For example, you can't copy instances as needed. The logic in Listing 2 w ill succeed.
an owned object from one entity instance to multiple other
classes. Listing 2 shows logic that attempts to create two It's interesting to compare the visualizations of the model
separate shipping label objects for one person. I t w ill fail. as well (using the wonderful EF Core Power Tool's diagram
tool) shown in Figure 1. As an Owned Entity, the DbCon-
Why? When this code assigns a person.Name to the second text sees the PersonName as its own entity in a one-to-one
label, EF Core moves i t from the Label i t was already as­
signed to. The first label w ill no longer have a Name— the
property is now null. In the docs, the EF Core team explains Mapped as Owned Entity
other common use cases that w ill fail as well. Keep in mind
that in unit tests that don't involve EF Core, the code poses
no problem and is sensible. It's EF Core's tracking that fails.
And as I said earlier, the team tried variations on how to
handle these types of problems across versions of EF Core,
all the while pondering how to implement a better map­
ping, rather than continuing to try to make owned entities
work across the various needed scenarios.

Mapped as ComplexProperty
Listing 2: Retrieving a Customer and Using its Name for a new Label Customer
var storedCustomer = ctx.Customers.First(); Customerld (inti J Name (PersonName) ||Hm
var label = new ShipLabel
{
Name = storedCustomer.Name, I 2 ' 1T
" ' ’ ■1
};
var label? = new ShipLabel
{ H ComplexType (Customer N.ime/'PrrsonName)
Name = storedCustomer.Name,
};
ctx.AddRange(label, label?); Figure 1: Data Model with Person mapped as an Owned
ctx.SaveChanges(); Entity vs. a Complex Property

34 Value Object's New Mapping: EF Core 8 ComplexProperty codemag.com

relationship with Customer. Notice that there's even a Cus- Listing 3: DebugView (LongView) with Name mapped as a ComplexProperty in Customer
tomerld shadow property. As a ComplexProperty, EF Core and ShipLabel_______________________________________
comprehends that it's just another property of Customer. Customer {Customerld: 1} Unchanged
Customerld: 1 PK
In addition to noting the differences between the model FirstPurchase: '1/1/2021'
in Figure 1, it's also interesting to see the DebugView Name (Complex: PersonName)
FirstName: 'John'
for the Customer and ShipLabel entities as they're seen LastName: 'Doe'
by the change tracker prior to calling SaveChanges in ShipLabel {Id: -2147482647} Added
Listing 2. Id: -2147482647 PK Temporary
Name (Complex: PersonName)
FirstName: 'John'
First is the view for the OwnsOne mapping, and there's so LastName: 'Doe'
much going on here th a t I'm only showing the ShortView. ShipLabel {Id: -2147482646} Added
Id: -2147482646 PK Temporary
Customer {Customerld: 1} Unchanged Name (Complex: PersonName)
FirstName: 'John'
Customer.Name#PersonName {Customerld: 1} LastName: 'Doe'
Unchanged FK {Customerld: 1}
ShipLabel {Id: -2147482647} Added
ShipLabel {Id: -2147482646} Added
ShipLabel.Name#PersonName up to do to learn about records because there are many
{ShipLabelld: -2147482646} Added FK ways to express them.
{ShipLabelld: -2147482646}

With the owned entity mapping, the Customer's Name and A Quick Records Overview
the Name of only one of the ShipLabels (remember, EF Records have a number of formats. I spent a lo t of time
Core moved it, not copied it), are tracked separately and understanding the various ways to express a record to
it's a b it convoluted. choose the correct flavor. The documentation was very
helpful (https://learn.m icrosoft.com /en-us/dotnet/
Listing 3 shows the DebugView when PersonName is csharp/language-reference/builtin-types/record), but it
mapped as a ComplexProperty: This time, I'm sharing the s till took a few read throughs for me. I have encapsu­
LongView with more details because it's so easy to read. lated some of the important details in Table 1 for a quick
The details look ju s t as you would expect. And EF Core is reference.
doing a lo t less work to manage the PersonName data.
To begin with, a record, by default, is a reference type.
It's much simpler and the side effects of the fake entities But a record struct is a value type— the correct choice for
ju s t disappear. a value object. There are more decisions to make.

You can define a record struct as a positional record,

Classes, Records, and Value Objects, which has nothing more than a primary constructor and
Oh My! looks Like this:
The PersonName value object shown in Listing 1 includes
a primary constructor to make i t simpler to instantiate. public record struct PersonName (
string FirstName, string LastName);
This Leads us to the first caveat of ComplexProperty. In
EF Core 8, i t won't work with a class th a t has a primary That's the entire implementation! Internally, C# infers the
constructor— emphasis on class. I f you want to map this FirstName and LastName string properties.
class with ComplexProperty, you need to remove th a t con­
structor and use an object initializer to instantiate a new Currently my PersonName record has no logic and is a
PersonName like this: good candidate for a positional record. I f you have no
need for logic or further constraints in the object, the
new PersonName{FirstName ="John",LastName="Doe"} streamlined positional syntax is awesome.

Otherwise, you'll need to go back to mapping with OwnsOne. But— and this is a big but— on its own, a record struct
is not, I repeat, not, immutable. Therefore, i t fails the
What about records instead of a class? I recall firs t seeing requirement of a value object. Luckily, C#12 added the
the exploration th a t the C# team was doing on records capability to make a record struct read only.
at an MVP summit quite a few years ago. Because of how
they simplified creating value objects, I was definitely public readonly record struct PersonName (
eager to see them come into the language. Record types string FirstName, string LastName);
internalize equality comparison so you don't need to
override the Equals or GetHasCode methods every single That's a very succinctly expressed and simple value object.
time.
I f you do need additional logic, you can express the re­
However, records did not play very well with owned enti­ cord more like a class with properties and other logic ex­
ties and again, there were side effects to worry about. plicitly defined, as I'm doing here, using in it accessors
Therefore, I never used records u ntil EF Core 8 brought us to ensure th a t it's s till immutable. This is an example of
the ComplexProperty mapping and I had a b it of catching a read-only record struct w ithout positional properties.

codemag.com Value Object's New Mapping: EF Core 8 ComplexProperty 35

 Declaration Type Mutability Mapped as an Owned Entity, EF Core can sort this out.
The default mapping results in Name_FirstName and
class Reference type Mutable unless designed otherwise
Name_LastName columns in the Customers table both be­
record Reference type Immutable ing nullable.
record struct Value type Mutable unless designed otherwise
The OwnedEntity mapping comprehends the null property
readonly record struct Value type Immutable
when I create and store a customer w ithout a name.
Table 1: How classes and records are interpreted
var customer=new Customer
{
public readonly record struct PersonName FirstPurchase=new DateOnly(2021, 1, 1)
{ };
public FirstName X { get; in it; } ctx.Customers.Add(customer);
public LastName Y { get; in it; } ctx.SaveChangesQ;

public string FullName => When Name is null, the Name_FirstName and Name_Last-
$"{FirstName} {LastName}"; Name database fields are both null as well. When I re­
} trieve the customer, EF Core returns a Customer with a
null Name property.
There's an interesting capability of records th a t you
should consider, which is th a t it's possible to create a new At some point, ComplexProperty w ill have the same be­
instance of a record with new values. This feature uses a havior. But currently (in EF Core 8), specifying Name as a
with expression to replace property values. nullable type results in a runtime exception. The exception
you get is dependent on how the value object is defined.
For example, I might have instantiated a PersonName using:
I f the value object is a class, you get a message about the
var jazzgreat=new PersonName("Ella", fact that i t can't be optional when EF Core is attempting
"Fi tzgeralde"); to build the data model based on the DbContext mappings.

Then I discover the typo of the "e" at the end of her System.InvalidOperationException: 'Configuring the complex
name. Of course, with only two properties, I could eas­ property 'Customer.Name' as optional is not supported,
ily create a new instance from scratch. But i f you have call ' IsRequired()'. See https://github.com/dotnet/efcore/
a lot of properties, you could use the with expression issues/31376 for more information.'
syntax:
Making i t required ju s t so EF Core is happy is not a pleas­
jazzgreat=jazzgreat with ing solution. I t should only be required i f your domain
{LastName = "Fitzgerald"}; invariants specify th a t Name should be required. I f it's re­
quired, you can use ComplexProperty. I f not, you're stuck
There are a lot of other nuances of records th a t you can with OwnsOne.
learn about in the docs at https://learn.m icrosoft.com /
en-us/dotnet/csharp/language-reference/builtin-types/ I f PersonName is a record struct (with or w ithout posi­
record. tional properties), you'll trigger a different exception. EF
Core configures the database fields from the value ob­
Because I found i t confusing to sort out all of the behav­ je c t properties (Customer_FirstName, and Customer_Last-
iors of the various flavors of record types, I've listed the Name) as non-nullable fields. At runtime, the database
critical aspects of each (as well as class for comparison) w ill throw an exception saying th a t i t can't insert a null
in Table 1. value into a non-nullable column.

Note: I f you Look into the GitHub issue referenced in the

Null Value Object Properties exception message, please add your vote to make sure
The most important caveat about EF Core 8's implementa­ the EF Core team addresses this. I do expect i t to be sup­
tion of ComplexProperty, which is a deal breaker for some, ported in EF Core 9 but every vote from the community
is th a t i t doesn't currently support null objects. We saw helps them prioritize.
this same problem with owned entities in an earlier ren­
dition, but owned entities now support null properties.
What Else Is and Isn't Supported?
As an example, Let's say I made PersonName nullable in Nullability of complex types, whether or not they are val­
the customer type. Perhaps this isn't a logical change, but ue objects, is an important topic, indeed. But there are a
i t serves my demonstration purpose. few other points to be aware of: some limitations as well
as things th a t are supported. Let's start with the good
public class Customer news about primary constructors.
{
public int Customerld { get; set; } Records and Record Structs with Primary Constructors
public PersonName? Name { get; set; } I said above th a t you can't map a class with primary con­
public DateOnly FirstPurchase { get; set; } structors using ComplexProperty. Well, happily, i t works
} with the records! You can map a ComplexProperty with

36 Value Object's New Mapping: EF Core 8 ComplexProperty codemag.com

the positional records (which are declared in their entire­ ComplexProperty Owned Entity
ty by a primary constructor) and non-positional records
Class Yes Yes
th a t have a primary constructor. I also successfully tested
this with positional record structs, positional read-only Record Yes With side effects
record structs, and non-positional record structs. I defi­ Record Struct Yes No (must be a ref type!)
nitely prefer primary constructors over expression build­
Record with Primary CTOR Yes Yes
ers to instantiate an object.
Class with Primary CTOR No Yes
JSON Support, or Lack Thereof, for Now Collections No (EF Core 9) Yes (OwnsMany)
Although JSON support has been improving in EF Core, Nested No (EF Core 9) Yes
some of i t thanks to Owned Entities, i t does not exist yet
Data Annotation available Yes Yes
for ComplexProperty.
Store in its own table No Yes
For example, i f PersonName is mapped as an owned en­ Map to JSON column No (EF Core 9?) Yes
tity, you can append the ToJson() method to the OwnsOne Seeding via DbContext/Migrations No Yes
mapping resulting in the object being stored in some type
Supported in Cosmos provider No Yes
of char field in your relational database table. A Person-
Name is stored as Table 2: EF Core 8 support for ComplexProperty and OwnedEntity mappings

{"FirstName " : "John", "LastName": "Doe"}

The question becomes (for some): Should you mix and SPONSORED SIDEBAR
ComplexProperty does not yet support this capability. match the two mappings? I think the answer is yes. I
Additionally, its inability to transform complex types to don't think i t needs to be seen as a maintenance prob­ Al Executive
JSON also means th a t you cannot use ComplexProperty lem. What ComplexProperty currently solves, i t does very Briefing
with the CosmosDb provider th a t stores all its data as well— and does so better than OwnedEntity. For the cases
JSON. Not yet. This is another feature th a t is tagged th a t you s till need to use Owned Entity, continue to use Experience the game­
in the GitHub repo as "consider for current release," so them. But keep an eye on those cases because you'll be changing impact of
hopefully th a t means EF Core 9. able to replace more (or all?) of those mappings when EF Al through CODE
Core 9 comes out. Consulting's Executive
Collections o f Complex Types: Coming Soon Briefing service. Uncover
Owned Entity not only provides the OwnsOne mapping, Table 2 provides you with a List of possible ways to define the immense potential
but also OwnsMany. Therefore, it's possible to have a complex types and value objects and whether or not that and wide-ranging
property in your entity that's a collection of the owned expression is supported with ComplexProperty and Owned benefits of Al in every
types. ComplexProperty doesn't yet support this, but the Entity mappings in EF Core 8. The EF Core team absolutely industry. Our briefing
provides strategic
team has said i t w ill be in EF Core 9. Keep in mind that plans for a near future when we can completely eliminate
guidance for seamless
value object collections are a disputed topic. Some call OwnsOne and OwnsMany from our code. Until then, take
implementation,
them an anti-pattern. But I've found some edge cases advantage of the to ol th a t works best for each scenario.
covering crucial aspects
where they are quite useful. In fact, I have a collection And test, test, test.
such as infrastructure,
of Author value objects in my EF Core and Domain-Driven
talent acquisition, and
Design course on Pluralsight. And even though I'm using The Limitations are listed in the docs at this link (h ttp s :// leadership.
EF Core 8 in the course, I s till had to map th a t particular le a rn .m ic ro s o ft.c o m /e n -u s /e f/c o re /w h a t-is -n e w /e f-
value object as an owned entity. Happily (and intention­ core-8.0/whatsnew#current-limitations). Each has a Link Discover how to
ally), the sample application in th a t course has another to the relevant issue on GitHub and you can Let the team effectively integrate
value object th a t provided a great example of using a know which are important to you by voting for these is­ Al and propel your
record and ComplexProperty mapping. sues in GitHub. organization into future
success.
Nested Complex Types: Also Coming Soon Julie Lerman
That Pluralsight course also demonstrates nesting value CODE Contact us today
to schedule your
objects. The Author value object has a PersonName prop­
executive briefing and
erty similar to the one I've been using in this article. Be­
embark on a journey
cause Author is already mapped as an owned entity, I had
of Al-powered growth.
to tack on its PersonName property as an owned entity
www.codemag.com/ai
as well. You definitely can't combine Owned Entities and
ComplexProperty when nesting.

More importantly, even i f Author was a ComplexProperty,

nesting is not yet supported in EF Core 8. In the end, not
only was I forced to use Owned Entity mappings for those
two value objects, because they were owned entities, I
had to declare them as classes, not records.

Is ComplexProperty Ready for

Your Software?
I'm very happy to see and use ComplexProperty. Although
it's not perfect yet, i t already does solve many scenarios.

codemag.com Value Object's New Mapping: EF Core 8 ComplexProperty 37

 ONLINE QUICK ID 2405051

Preparing for Azure with

Azure Migrate Application and
Code Assessment
There are many advantages to hosting ASP.NET and ASP.NET Core applications in Azure. Platform-as-a-Service (PaaS) environments
like Azure App Service, Azure Kubernetes Service, and Azure Container Apps provide easy and automatic scalability, reliability,
and availability in multiple geographies. What's more, these environments allow you to focus on the apps themselves without

the overhead o f m aintaining underlying infrastructure. SDK to o l. The to o l scans ASP.NET and ASP.NET Core solu­
But fo r apps th a t are already deployed on-premises, i t tion s (and, optionally, th e ir binary dependencies) fo r a
can be d iffic u lt to know how to get started re-p la tform ­ wide variety o f p o te n tia l issues th a t need to be addressed
ing to one o f these environm ents. Even though .NET ap­ prior to running in Azure PaaS environm ents.
plications can often by deployed to Azure App Service
w ith m inim al changes, there are usually some changes Although th is article focuses on the experience o f using
required and discovering w h at those are can take some Azure Migrate application and code assessment fo r .NET,
tr ia l and error. there is also a Java version o f the to o l available. To Learn
more about the Java experience, please v is it h ttp s ://le a rn .
Mike Rousos This article introduces a new feature: Azure Migrate ap­ m icrosoft.com /azure/developer/java/m igration/appcat.
[email protected] p lica tion and code assessment. This new feature allows
analyzing the source code, con figu ratio n, and binaries
Mike Rousos is a Principal o f an application to discover upfront w hat changes w ill
Relationship to Other Azure Migrate
Software Engineer on the be needed fo r the app to work in Azure. Azure Migrate Features
.NET Customer Engage­
application and code assessment make i t easy to plan Using Azure Migrate to prepare fo r a m igration to the
ment Team. A member of
re-platform ing to Azure and high lig hts w hat work w ill be cloud is n 't new, o f course. Azure Migrate has helped users
the .NET team since 2004,
needed along the way. Azure Migrate application and code discover and assess on-premises infrastructure fo r some
he has worked on a wide
assessment is a developer-focused experience available as tim e. Azure Migrate also includes the Data M igration As­
variety of feature areas and
both a Visual Studio extension and a command Line .NET sista n t to help users assess SQL Server databases fo r m i-
contributed content to the
.NET team blog, .NET Conf
sessions, Channel 9 videos,
*■1
and .NET development e-
books Like "NET Microser­
0
vices: Architecture for
Containerized .NET Appli­ Visual Studio | M a rke tp la ce Mika Rouwt (mikatou^rracrtnofLcom) Sign out

cations." Outside of work,

jie Migrate a ftp
Mike is involved in his
church and enjoys reading,
writing, and games of all
sorts. His primary hobby, Azure Migrate application and code assessment
though, is spending time M icrosoft O rn ic ro s o ft.c o m | 112 installs | ★ ★ ★ ★ ★ ( ! ) I Free
with his four kids.
Azure Migrate application and code assessment for .NET helps to assess .NET applications source
code, settings and dependencies to identify replatforming and migration opportunities for Azure.

O vervie w Q & A R ating & Review

Azure Migrate application and code assessment for .NET C a te g o rie s

Tools Coding Programming languages

The goal of this extension is to assist in migrating your .NET applications to Azure by
Web
performing a scan of the NET source code, configurations and binaries o f your
application to identify any issues your application might have when it is ported to Azure
Tags
App Service. AKS or Azure Container Apps. The tool also gives you recommendation^
when applicable, on how to improve the performance, scalability and security of your AppCAT ASP.NET Azure Cloud

Figure 1: Azure Migrate application and code assessment extension download page

38 Preparing for Azure with Azure Migrate Application and Code Assessment codemag.com
gration to Azure SQL DB, Azure SQL Managed Instance,
Search Solution Explorer (Ctrl +;)
or SQL Server on an Azure VM. Azure Migrate can even
a S Solution ‘eShop Leg acyM VC (3 of 3 projects)
discover web apps hosted on-premises and ( if no blocking
issues are detected) autom ate m igrating them to Azure r> ft eShopLegacy.Common
w ith its App Service M igration Assistant to o l. > fi eShopLegacy.Utilities
— U d U y ivi v
Up u n til now, though, Azure Migrate hasn't had the a b ility tew Build
lected Services
to look a t the source code o f the applications i t detected. Rebuild lerties
The App Service M igration Assistant to o l can provide in ­
Clean fences
s ig h t in to w hether there are likely to be issues m igrating
View ► _Data
to Azure based on IIS con figu ratio n, b u t i t doesn't have
v is ib ility in to what's happening inside the code o f the Analyze and Code Cleanup ► .Start
ap plication. The Azure Migrate application and code as­ & Publish-. ent
sessment feature fills th is gap. The Visual Studio exten­ rollers
sion or command line to o l can assess your source code o f Upgrade
5
your solution and id e n tify p o te n tia lly problem atic APIs 1 . Re-platfcrm to Azure
jes
and code patterns. This assessment is a useful follow -up ** Configure Application Insights...
els
to the discovery by oth er Azure Migrate features. I t is
recommended to use Azure Migrate's existing application Overview ifrastructure
i r r n i m tV io iA /M o r io le r e
discovery features to gain in s ig h t in to the com plete set
o f applications to be m igrated. Then, a fte r the applica­ Figure 2: The Re-platform to Azure command opens the new to o l UI.
tion s have been reviewed and prio ritize d , Azure Migrate
application and code assessment can be used by devel­
opers to dive deeply in to the applications th a t w ill be AppCAT: Hom e -o X

moved to Learn more details about p o te n tia l issues and to

plan the m igration. Azure M igrate application and code assessment fo r .NET Leave Feedback
Find out how your projects and their dependencies are com patible w ith Azure

Installing Azure Migrate Application

and Code Assessment © Select projects
As a Visual Studio extension, the Azure Migrate applica­
tio n and code assessment feature is easy to in s ta ll. It's
Next
available on the Visual Studio Marketplace a t h tt p s : / / Select projects to analyze
m arketplace.visualstudio.com /item s?item N am e=m s-dot-
Selected projects and their dependencies will be analyzed tor compatibility with Azure
nettools.appcat, as shown in Figure 1. From th a t site,
use the download bu tto n to download the .vsix installer, J I/ eShoplegacyMVC
and then open the in sta lle r and w a it w hile i t in sta lls the eShopLegacy.Common
Visual Studio extension. I0 eShopLegacy.Utilities

✓ •] eShopLegacyMVC
Note th a t the Azure Migrate application and code assess­
m ent Visual Studio extension requires Visual Studio 2022.
I f you do n 't have Visual Studio 2022, the free commu­ Figure 3: Selecting projects to be analyzed fo r Azure m igration readiness
n ity e d itio n is available a t h ttp s ://v is u a ls tu d io .m ic ro s o ft.
com /downloads.
also binary dependencies, as shown in Figure 4. Choos­
in g to analyze source code w ill Look a t th e source code
Analyzing a Solution o f th e selected pro je cts— C# or Visual Basic code file s,
To begin analyzing a solutio n, open the solution in Visual p ro je c t file s , con fig file s , and s ta tic co n te n t. This op­
Studio and rig h t-c lic k on i t in the solution explorer. W ith tio n should ty p ic a lly be checked. I f you also check the
the Azure Migrate application and code assessment exten­ Binary dependencies o p tio n , th e to o l also analyzes
sion installed, there w ill be a new command available: binaries th a t th e projects depend on. This includes re f­
Re-platform to Azure, as shown in Figure 2. Choosing erences to Loose DLLs, references to assemblies (o th e r
th is op tion opens a new user interface fo r managing tha n .NET Framework com ponents) in th e g lo b a l assem­
Azure Migrate application and code assessment reports. bly cache, or referenced NuGet packages. Choosing to
analyze binaries produces th e m ost com prehensive set
A fte r clicking New Report, you w ill be able to choose o f issues th e a p p lica tio n may run in to w h ile m ig ra tin g ,
which projects in the solution you wish to assess, as b u t i t also includes more issues tha n analyzing on ly
shown in Figure 3. Note th a t the Azure Migrate applica­ source code and may include issues th a t you're n o t able
tio n and code assessment feature autom atically analyzes to fix d ire c tly (because th e y e xist in com ponents you
dependencies o f selected projects, so you only need to d o n 't have source code fo r) or th a t a re n't relevant i f
choose to p -le v e l projects you're interested in . A ll recur­ th e y e xist in code paths n o t used from you r a p p lic a tio n .
sive p ro je c t-to -p ro je c t dependencies o f the selected pro j­ A useful strategy is to begin on ly by analyzing source
ects w ill be included autom atically. code and Later consider producing another re p o rt w ith
binary analysis enabled i f there are binary dependencies
A fte r c lickin g next, the fo llo w in g UI w ill allow you to whose Azure-readiness you're unsure o f and would like
choose w hether to scan o n ly source code and settin gs or to learn more about. Because issues in binary dependen-

codemag.com Preparing for Azure with Azure Migrate Application and Code Assessment 39
 cies can't be fixed directly by updating source code, they The report's dashboard includes the to ta l number of proj­
are typically addressed by finding updated versions of ects scanned, the number of incidents discovered, and
the binaries, working with partners who can change the graphics showing the incidents by category and severity.
source code, or finding alternative solutions th a t work The report include both a number of issues th a t are the
better in Azure. types of problems detected and a number of incidents
th a t are the individual occurrences of the issues.
Once you click the Analyze button, the extension analyzes
the selected projects for any potential issues re-platform­ Azure Migrate application and code assessment issues are
ing to Azure. This analysis w ill take anywhere from a few each assigned one of four severities:
seconds to a few minutes, depending on the size of the
projects. When the analysis is complete, you'll be shown 1. Mandatory: Mandatory issues are those th a t likely
a report summarizing the results. (See Figure 5.) This re­ need to be addressed before the application w ill work
port can be saved to disk and returned to Later using the in Azure. An example of a mandatory issue is using
save icon (or common shortcuts Like Ctrl+S). Windows authentication to authenticate web app us­
ers. Because th a t authentication mechanism depends
on the on-premises Active Directory environment, i t
w ill Likely need to be updated in the cloud to use
Azure AD or some other authentication alternative.
2. Optional: Optional issues are opportunities to im­
prove the application when it's running in Azure, but
they aren't blocking issues. As an example, storing
app settings or secrets in a web.config file is consid­
ered an optional issue. That pattern w ill continue to
work when deployed in Azure, ju st like on-premises,
so no changes are required. Apps that are hosted in
Azure can take advantage of services like Azure App
Configuration and Azure Key Vault to store settings
in ways that are easier to share and update and that
are more secure. So, there's an optional issue to be­
gin taking advantage of these services as part of the
Azure re-platform.
3. Potential: Potential issues represent situations where
there might need to be a change made for the app to
work in Azure, but it's also possible that no change is
needed, depending on the details of the scenario. This
Figure 4: Choosing whether to analyze source code or binaries severity is common and requires an engineer to review

Figure 5: The Azure Migrate application and code assessment report dashboard

40 Preparing for Azure with Azure Migrate Application and Code Assessment codemag.com
 the incidents. As an example, connecting to a SQL viewing incidents for a particular project, you can choose
Server database is a potential issue because whether to view all incidents for the project or to view incidents
a change is needed depends on whether the database per component (a single source file or binary dependency
that's used is accessible from Azure. I f the database is considered a component).
is already hosted in Azure or is accessible from Azure
(via Express Route, for example), no changes are need­ In incident detail views, there w ill be a state drop-down
ed. On the other hand, if the database used exists box indicating whether each incident is Active, Resolved,
on-premises without a way for the app to connect to or Not Applicable (N/A). ALL incidents begin as Active
i t once it's running in Azure, thought w ill need to be and you can change the state as you investigate. Reports
given to how this dependency w ill work after the app can be saved (using the save icon in the top right of the
is re-platformed. Perhaps the database w ill need to be report) and the state w ill be persisted so th a t you can
migrated alongside the app or perhaps a solution like return to the same report in the future and continue to
Express Route or Hybrid Connections w ill be needed to review remaining issues and further update the state. As
make the database accessible. you review the incidents in the report, mark incidents
4. Information: Information issues are useful pieces th a t don't need to be addressed in your solution as not
of information for the developer to know but don't applicable and those th a t you've fixed as resolved. The
require any action. As of the time of this writing, incident detail pages also give descriptions of why the
there aren't any information issues in Azure Migrate incidents were identified, why they matter, and how you
application and code assessment for .NET (although can address them. These detail views, shown in Figure 6,
there are a couple in the Java version of the tool). include Links to documentation and Links to the locations
in source code where the issues were detected.
In addition to severity, each incident in the report in ­
cludes a story point number. This is a unitless number In addition to working with reports in the Visual Stu­
representing the relative effort estimated to address the dio IDE, it's possible to export the reports to share with
incident (if, in fact, i t needs to be addressed). These others. The Export button in the top right of the report
shouldn't be used to estimate the precise amount of work interface allows you to export the report in three differ­
in terms of hours or days but can be used as a rough ent formats:
estimate for comparing two projects. I f one solution has
500 story points worth of issues and another has 200 • Export as HTML produces the most readable report for
story points worth of issues, it's probably true th a t the sharing with others. The HTML report, shown in Figure
solution with fewer story points of issues w ill be simpler 7, has all the same dashboards and views as the Visual
and easier to re-platform. Studio UI and includes snapshots of the Latest state of
all incidents. This report is best for sharing with others
From the in itia l dashboard, you can navigate to views dis­ for viewing issues and investigation progress.
playing aggregate issues (all incidents organized by issue • Export as CSV produces a report with the same in ­
type) or projects (incidents organized by project). When formation but in a spreadsheet format. As seen in

Azure Migrate application and code assessment for NET

Azure Compatibility Report

U 0 Lqxxt

• eShopLegaCyMVC.CSproj (C\xc\mpou»os\UpgradeSaznp‘<‘V>et4??\eShopiegaQMVC)

L 'd ititx ix d L cm p o o w iti h w r .

stuet
COMPONENT STATE K IN D IS S U E S ® IN C ID E N T S © STOEY P O IN T S ®

R te 1 2 B

You are uting M SM Q that * not current!/ cupported m Azcre App

issue msc r ipt io n st at e
ServxelAety You can u m Azure Queue Storage or Service But queue*
rn 'e a d at M > M (j A noihn option n to r u i M SM Q n Azure Virtual
Q u o u a 0003 M SM Q >< - i» i« 1 Al * A^tve • Machine*

Oweve.0003 MSMQ usage n detected (8 < ' Act w Source

uung ivai queue • new M eitageO ueue

(CoofigurattonManager AppSett,ngc[ NewltemQueuePath ’]))
|
war message • new Message
<
formatter • new Xm M esiag ef or matter (newfl ( typeo!
(Catalogitem |i
Body ■ catatogMem
label • 'New catalog item*
J:

••e m x <w w b v

Figure 6: Azure Migrate application and code assessment issue details

codemag.com Preparing for Azure with Azure Migrate Application and Code Assessment 41
 line, meaning th a t analysis doesn't need to depend on
Visual Studio and can be scripted, i f needed.

Like all .NET SDK tools, the Azure Migrate application and
code assessment CLI to ol is installed using a .NET CLI
Azure Compatibi ity Report
command:

D **^ *^ C 4
S U M M A XV C A ItU V K Iti dotnet tool install -g dotnet-appcat
M
* ________ 9 That command pulls down a NuGet package containing
a______ _______ 9
,______ . . . 9 the latest version of the dotnet-appcat to ol and installs
* _____ _ _____ 9
a______ _______ 9 i t globally. The to ol is run using the appcat command.
,______ . . . 9
a_____ _ i _____ 9 The simplest way to use the Azure Migrate application
a_____ . . . _____ 9
a______ ___ 9 and code assessment CLI is to run appcat analyze solu-
. ________ 9
* 9 tion.sln where solution.sin is the path to the solution or
* _____ _ _ . _ 9
a . „ ___ . . . ____ 9 project file you want to assess. Running analyze without
. ______ _ 9
. _____ _ _ . _ 9 a project or solution specified causes the to ol to search
a_____ _______ 9
. ______ i______ 9 for projects in the current directory. This command starts
a_____ - a . ____ 9
a_____ _______ 9 an interactive command line experience, allowing you to
___ . . . _ 9
a _____ 9 choose which projects from the solution you want to as­
a ______ _ _ _ 9
a_____ _ i _____ 9 U U H ’ sess, whether you want to analyze only source code or
a _____ - i ____ 9
a_____ . - a ____ 9 source code and binaries, and what output format you
: ig u t F l t ) f“Assessr^engresults for the eShop sample’ want (HTML, CSV, or JSON)— all the same options as with
a______ - u ___ 9 ~
a ______ 9 the Visual Studio extension! See Figure 12.
a ______ 9
A___ — a - ____ 9
*- ■ i -I r* ii LaiMB 9
Q Those are all of the etected in the eShop sample. Once the appcat CLI to ol gathers the necessary data from
a . __ 9
a______ 9 It's a larger List tha Uy expected, perhaps, but, as you, i t proceeds with analysis and publishes results to a
9
a _______ 9 we thought, the-iss e almost entirely about exter- report using your desired format.
» - - ~ L . . MM 9
a_____________ 9 nal dependencies-th need to be migrated to Azure
a______ ___ 9
, ______ _ .____ 9 along with the eSho ion. There was nothing block- One important note about the CLI experience is th a t
a______ ____ 9
a_. - a 9 ing in the report-exc the helpful reminders th a t the the solution must be able to build w ithout errors. Visual
,________ _____ 9
a _____ ___ 9 solution w ill need-d , message queue, and caching Studio can provide the necessary symbol information for
a_____________ 9
a_____ . - a ____ 9 solutions in the-elo may look different from on- analysis, but when running from the command line in ­
a________ _____ 9
a______ ____ 9 premises, and previ those (and updating the app stead, the target project must be in a buildable state.
a_. - a 9
._______ 9 to use them) needs Su*** 111
art of the re-platforming plan.
a_____ - I . ____ 9
a_. - a 9 In addition to mimicking the Visual Studio experience,
* n h — ... . L — 9

a______ - a . ___ 9 tu n the Azure Migrate application and code assessment com­
a______ ____
a______ . - a ___
9
9
Mun
Using the Co nd Line Interface mand Line to ol accepts parameters th a t allow all decisions
,________ _____ 9 buei
a_____________ 9 In addition to the tudio extension discussed in to be made up-front so th a t the to ol runs completely
a_____ - a____ 9
a _______ 9 the rest of this artic
IJ I
Azure Migrate application and automatically, allowing for a non-interactive experience
a______ ____ 9
* _____________ 9 code assessment fea also available via a .NET com- suitable for scripting. To do this, use the -non-interac­
a______ ___ 9
a________ _____ 9 mand Line tool. Th L Studio extension offers the tive parameter and make sure to specify report format
a______ ____ 9
. - L .._ 9 most functionality e the reports maintain state and components to analyze via the command line. Here's
.______ . . . _____ 9
a_____ a - I.____ 9 for which incidents en reviewed), but all the same an example command Line for assessing a C# project non-
a_. - a 9
a _______ 9 assessment capabiliIJI available from the command interactively:
a_____ - a . ____ 9
a_____ — a.____ 9
_______ ___ 9
._______ 9
a_____ . . . ____ 9
a_. . a 9
.______ . . . ____ 9
a_____ .. a . ____ 9
a_____ - u ____ 9
_______ ___ 9
._______ 9
a_____ . . . _____ 9
a_. . a 9
* n h — « ... L — 9

a _____________ 9
a_____ - u ____ 9
>. a^aw . .t e a . Mte 9
v

a _____ . . . _____ 9
a_____________ 9 4UX5
_ 9 t u n ■
» - ■ —________________9
a______ . . . ____ 9
a_ . . a 9
a_ . . . 9
a ______ ____ 9
a_____ . . . _____ 9
a______ ___ 9
._______ 9
a_____ . . . _____ 9
a_ . . a 9
a_ . . . 9
a _____ . . . _____ 9
a_____ — i____ 9
a_____ . - a ____ 9 tim et C
. _____ . . . _____ 9
a_____ . . . _____ 9
a_ . . a 9
a_ . . . 9
a_____ . . . _____ 9
a_____ . - a ____ 9
: ig u te :llrR a bb ith i(t'related
I issues in eShop
9 Q
9 num Q
9 u u m Q
9 u m Q

7
Preparing for Azure with Azure Migrate Application and Code Assessment codemag.com
a_____ _______ 9 iita m O
9 " |1|
9 * '«u<% {7) (1)
appcat analyze eSliopLegacyMVC.cspro] -s HTML
-r OutputPath --cade --non-interactive

This command assesses the eShopLegacyMVC.csproj proj­

ect, only consider source code (thanks to -code), and
puts an+JTML report in the OutputPath folder. The output is
the same as the Azure Migrate applisation and code assess­
ment Visual Studio extension .produced and i t all runs au­
tomatically from the command Line as shown in-Figure IB.

Road Map
The Azure Migrate application and code assessment feature
alraadyqjrovides insights necessary to plan a successful Aaure
migration. In theYuture, though, there are even more useful
features planned. Key features coming in future versiens of
Azure Migrate application and code assessment include:

H. -CttHub CopMot Chat in te c ’ation: Azure Migrate ap­

plication and code assessmerft is great at identifying
points of interest in an application th a t should be
Figure U c Azure Migrate application and code assessment command line output
reviewed to be prepared for re-platforming to Azure.
It's only an analysis tool, though, and guidance
that's given on how to address issues is static. -By
partnering with CitHub Copilot Chat, i t w ill be pos­
sible for users to have reports from Azure Migrate ap­
plication and.code assessment summarized for them
and to have back-and-forth conversations about the
best way to review and remediate issues. GitHub Co­
p ilo t Chat w ill be knowledgeable about the issues
discovered by Azure Migrate application and code
assessment and w ill be able to provide stepj, byj step
instructions for configuring Azure resources, updat­
ing Gode, and more, to address them.
2. Multiple Azure taigets: A feature available for the
Save version of Azure Migrate application and code
assessment that isn't yet available for the .NET ver­
sion of the tool is the ability to specify a desired
Azure target environment during analysis ffor exam­
ple, Windows Azure App Service or AKS with a4_inux
container). The issues th a t apply to different Azure
environments overlap a Lot and, cuirently, Azure Mi­
grate application and code assessment Tor .NET-as­
sesses a general Azure PaaS environment th a t covers
issues of interest in any potential target. "But this
feature w ill be coming soon to Azure Migrate ap­ Figure The Azure Migrate application and code assessment CLI offers the same
plication and eode assessment for .NET. In the next options as the Visual Studio extension.
version, you w ill be able to specify an Azure environ­
ment to target and issue severity levels and descrip­
tions w ill be updated for th a t environment. You can learn more about Azure Migrate application
B. ^Binary output analysis: Today, Azure Migrate applica­ and code assessment from documentation available at
tion -and code assessment can scan binary dependen­ https://learn.microsoft.com/aaure/migrate/appcat/over-
cies referenced by a solution. But there s till has to be view. As you use the new feature, please send feedback i f
a source fjolution as a starting point. In the next ver­ you have ideas for how the experience Gould be improved
sion Of the tool, the command line interface w ill allow or i f you run into any .bugs. The feature is brand new,
scanning compiled binaries in a given folder directly. so there are bound to be opportunities to improve i t as
customers use the Visual Studio extension and CLI and
share their experiences. To get instructions on how to
Wrap-blp leave feedback, click the Leave Feedback link in the top
Azure Migrate has been helping developers migrate so­ right corner of the Azure Migrate application and code
lutions from on-premises to Azure for years. With the assessment's Visual Studio UI or use the Q&A tab of the
addition of the Aaure Migrate application and code as­ Visual Studio marketplace page at https://marketplace.vi-
sessment feature, th a t migration help new extends to sual$tudio.com/>tems?itemName=ms-dotnettools.appcat.
understanding the inner workings of your solution and
source-level changes th a t are needed for fhe application Mike Rousos
to work in Azure platform-as-e-service environments! IJ L L

codemag.cQm Preparing for Azure with Azure Migrate Application and Code Assessment 45
ONLINE QUICK ID 2405061

Stages of Data: The DNA of a

Database Developer, Part 1
Long before I started in IT, I wanted to teach. The process of learning something, then performing it, crafting it, learning from
mistakes, further crafting it, and sharing that journey with others appealed to me as a great career. Rod Paddock (Editor-in-
Chief for CODE Magazine) recently told me that the next issue of CODE would deal with databases. At the same time, I've been

seeing many questions on Linkedln th a t b o il down to , ticle . This w ill be a tw o -p a rt article. Throughout both,
"How do I increase my SQL/database skills to get a data I'm going to m ention some topics th a t I covered in prior
analyst/database developer jo b ? " In th is industry, th a t CODE Magazine articles where the con ten t is s till ju s t as
question is com plicated, as i t means d iffe re n t th in g s to relevant today.
d iffe re n t people. It's arrogant to claim to have a ll the
answers, because doing so would presume th a t someone You can fin d many web articles w ith title s Like, "Here are
knows about every jo b requirem ent o u t there. Having said the 30 best SQL Server in te rvie w questions you should be
th a t, I've worked in th is industry fo r decades, both as an prepared fo r." There are many good ones and I recommend
employee and as a contractor. I'd Like to share w h at skills reviewing them as much as you can. I also recommend
Kevin S. Goff have helped me get and keep a seat a t the table. g e ttin g your proverbial hands d irty inside o f M icrosoft
www. Kevi nSGoff. net SQL Server Management Studio. On th a t note, I'm using
@StagesOfData M icrosoft SQL Server, which means I'LL be covering some
Opening Remarks: M icrosoft-specific topics. Having said th a t, many o f the
Kevin S. Goff is Database A Rebirth o f SQLServer Skills topics in th is article are relevant to other databases.
architect/developer/
Over the last few years, there have been intense opportu­
speaker/author, and
nities and jo b growth in the general area o f data analyt­ I d id n 't w ant to call th is article "The 13 things you should
has been w riting for
ics. That's fan ta stic! There's also been a re a lity check o f study before a SQL interview ," because I'm going beyond
CODE Magazine since
som ething th a t database professionals warned about last th a t. I'm covering w hat I th in k makes for a good SQL/
2004. He was a member of
the Microsoft MVP program decade: the need fo r those using self-service BI tools to database developer. Yes, there's overlap, as I w ant to share
from 2005 through 2019, have some basic SQL and data management skills. As part some o f the specific skills companies are often looking for.
when he spoke frequently o f research, I spent a substantial am ount o f tim e reading
community events in the Linkedln posts, and ta lk in g to recruiters and oth er devel­
Mid-Atlantic region and opers about th is , and there's one common them e: There's First, Know Basic SQL
also spoke regularly for s till a boom ing need fo r SQL and data handling skills. I "Know ing basic SQL" is really tw o th in g s: understanding
the VS Live/Live 360 w ant to make a jo ke about being an o ld -tim e SQL person the SQL language (according to the ANSI SQL standard)
Conference brand from and a "boom er," b u t I was born one m onth a fte r the o f­ and understanding specific features in the database prod­
2012 through 2015. fic ia l end o f the boomer generation. uct (in th is article, M icrosoft SQL Server) and some o f the
physical characteristics o f M icrosoft databases. There are
I'm a big sports and music fan and I often hear people talk great books o u t there, b u t these topics tend to come up
about the "DNA" o f great athletes and musicians. In this again and again. I 'l l s ta rt w ith some index basics, even
context, the definition isn 't referring to the biological as­ before g e ttin g in to some language basics.
pects o f a person. It's more the traits they carry w ith them
and the habits they've burned in to themselves th a t they K now the D iffe re nt Types o f Indexes
leverage regularly to do th e ir jobs and do them well. I cer­ A common question is the difference between a clustered
ta in ly hope th a t everyone who's w illing to work hard w ill get index and a non-clustered index. W ith th is and other to p ­
a jo b, and I'm equally excited th a t I've seen a resurgence of ics in th is article, I'm n o t going to w rite o u t a fu ll d e fin i­
"w hat SQL Server skills should a data person have?" tio n , because oth er websites have done a great jo b . But
here are th in g s I th in k you should know.
I know people who build great websites and great v i­
sualizations in reporting tools, where the available data You can only create one clustered index per tab le and
was very clean and prepared by an existing data team . th a t index defines the sorted order o f the table. For a
However, fo r every one in d iv id u a l jo b o u t there like th a t, sales table th a t m ig h t have m illions o f rows, a clustered
there's more than one jo b where y o u 'll have to p u t on index on e ith e r the sale transaction ID or possibly the
your SQ L/data-handling hat. sales date w ill help queries th a t need to scan over many
rows in a pa rticular order.
I've worn m ultiple hats in the sense th a t I've always had
work. I make mistakes, I underestimate, I s till com m it You can have many non-clustered indexes. They serve
many s illy errors th a t we all, as developers, wish we could the purpose fo r more selective queries: th a t is, fin d in g
avoid. A lthough no one person can possibly cover every sales between tw o dates, fin d in g sales fo r specific prod­
possible SQL/data s k ill th a t w ill make someone success­ ucts, specific geographies, etc. A non-clustered index
fu l, I stepped back and th o u g h t, "W hat has helped me m ig h t contain ju s t one column (com posite index), or i t
to help clients? What has been the difference-m aker on could contain m u ltip le columns i f y o u 'll frequently need
a project?" And th a t's why I decided to w rite th is ar­ to query on a com bination o f them .

Stages o f Data:The DNA o f a Database Developer, Part 1 codemag.com

 Table Scan
Query 1: Query cost (relative to the batch): 1001 Scan rows from a table.

SELECT • FROM (TestFerson] WHERE (BusinessEntitylD]«81

P h y » K il O p eratio n Table Scan
lo g ic a l O p eratio n Table Scan
A ctual Execution M o d e Row
E stim ated Execution M o d e ________ Row
S torage _____________________________ RowStore
A ctual N u m b e r o f Rows Read 19972
A c tu a l N u m b e r o f Rows fo r AH Executions ________________ 1
A ctu al N u m b e r o f B a tc h e s ______________________ ________________0

Figure 1: Execution plan, only the execution operator is a Table Scan w ith statistics on the number o f rows read

G uttered Index See* (G uttered)

Query 1: Query cost (re la tiv e to the b atch ): 1001 Scanning i particular range of rows from a clustered m pei
SELECT * FROM (TestFerson) WHERE [B usinessE ntitylD ]=81
id* Physical Operation Clustered index Seek
Logical Operation Clustered Index Seek
C lustered Index Seek (Claate
Actual Execution Mode ______________ Row
(TestFerson]. [xxBusm essEnc.
tSllHUiCO tXCCUiKMi MOOC __________ Row
C on : 100 i
0.000a Storage _________ RowStote
Actual Number of Rows Read a
1 of
4
I (100«) Actual Number of Rows for A l Executions
Actual Number of Batches 0

Figure 2: Execution plan w ith an INDEX SEEK, which is fa r more e ffic ie n t (on ly one row read)

Related: K now When SQL Server W ill Use These Indexes: I f I run the same query again and then Look a t the execu­
Ju st because you create an index doesn't mean SQL Server tio n , I'Ll see a very d iffe re n t story: a INDEX SEEKCLUS-
autom atically uses it . For instance, you m ig h t create in ­ TERED INDEX SEEK where SQL Server only needed to read
dexes th a t SQL Server w o n 't use, e ith e r because the SQL one row (Figure 2).
statem ent you're using is n 't search argum ent optim izable
or because you m ig h t n o t realize w hat som ething like Next, I'Ll query the table fo r a ll names where the last
compound indexes w ill (and w on't) do. name is "Richardson".

I'm going to take the table Person.Person from the Ad­ select * from testperson
ventureWorks database and create my own table. I 'l l also where lastname = 'Richardson*1
create tw o indexes: a clustered index on the prim ary key
(BusinessEntitylD) and a non-clustered index on the Does the clustered index help us a t all? U nfortunately,
Last Name and the First Name. n o t really. A lthough SQL Server scans a clustered index
instead o f a row table (heap), SQL Server must scan a ll
drop table i f exists dbo TestFerson 19,972 rows (Figure 3 ).
go
select * into dbo TestFerson from Person Person To help w ith queries based on a last name, I'Ll create a
non-clustered index on the Last Name column.
Now I 'l l use a single query to retrieve the row fo r a spe­
c ific Business E ntity ID: create nonclustered index [ix_LastName]
on TestFerson ( LastName)
select * from TestFerson where BusinessEntitylD
= 12563 A fte r creating the index on Last Name, let's query fo r a
specific Last name, and then fo r a specific Last name and
In the absence o f any index, SQL Server must perform a firs t name:
table scan against the table and read a ll 19,972 rows.
Here's w hat SQL Server returns fo r an execution plan (Fig­ select * from testperson
ure 1). where lastname = 'Richardson1

A lthough the query runs in a s p lit second, SQL Server had select * from testperson
to read through a ll the rows. It's n o t exactly optim al. where lastname = ‘ Richardson’ and
firstname = 'Jeremy'
Now Let's create a clustered index th a t drives the sort
order o f the table: In the case o f the firs t query, SQL Server uses a more e ffi­
cie nt INDEX SEEK on the new index. However, i t does need
create clustered index to perform what's called a KEY LOOKUP in to the clustered
[ix_BusinessEntityClustered] on TestFerson index, to retrieve a ll the columns (because I did a SELECT
(BusinessEntitylD) * to ask fo r a ll the columns).

codemag.com Stages o f Data:The DNA o f a Database Developer, Part 1 47

 Clustered Index Scan (Clustered)
Query 1: Query cost (re la tiv e to the batch): 100%
Scanning a clustered index, entirety or only a range.
SELECT • FROM (testperson) WHERE [lastname)=81
4} Physical Operation Clustered Index Scan
Logical Operation _____________________ Clustered inde* Scan
Clustered Index Scan (Cluste Actual Execution Mode _________________ Row
[TestPerson]. [xx_Business£nt ts iim iic o execution mo o c Row
Cost: 100 I Stof f p ____________ RowStore
0.006s Actual Number of R o m Read _________________ 19972
91 of Actual Number of Row* for All Executions 91
93 (971) Actual Number of Batches _____________________0

Figure 3: Execution Plan, now showing a Clustered Index scan on a ll 19K rows

To achieve the same general optim ization, you'd need to

create a separate index where the leftm ost column (or only
N e ste d L oops In d e x S can (N o n c lu s te r e d ) column) is the firs t name. Bottom Line: SQL Server reads
(In n e r J o in ) [ T e s t P e r s o n ] . [lx _ L a s tN a m e F ir
the Leftmost key columns, so keep th a t in mind when de­
C o st: 4 9 C o s t: 40 %
0 .0 0 5 s 0 .0 0 3 s signing indexes, and query against m ultiple columns.
47 o f 47 o f
4 2 (111% ) 4 2 (111% ) You can fin d many websites th a t ta lk about com posite
indexes. Here's a pa rticularly good one: h ttp s ://le a rn .
m ic ro s o ft.c o m /e n -u s /a n s w e rs /q u e s tio n s /8 2 0 4 4 2 /s q l-
K ey L o o k u p ( C l u s t e r e d ) server-w hen-to-go-for-com posite-non-cluster-in
[T e s t P e r s o n ] . ( i x B u sr n e ssE n t^ .
C o s t: 55 % Indexes and Fill Factors
O.OOOs
Some developers who're very good a t SQL queries hesitate
47 o f
4 2 (111% ) to get involved w ith questions th a t get closer to w hat
some m ig h t norm ally fe e l is "th e jo b o f a DBA." As an ap­
Figure 4: Execution Plan, and you're back to a scan, because the query couldn't use the index plications developer, I'Ll freely ad m it th a t I d o n 't have fu ll
knowledge o f a DBA. However, there are *som e* topics
th a t cross over in to the application space, and even i f you
W hat happens when you query both a firs t and Last name? do n 't use them , it's s till im p o rta n t to a t least understand
You only retrieve one row, so the server does Less work why some tasks are n o t perform ing w ell.
to return the results back to the application. However,
under the hood, you'd see som ething d iffe re n t. Yes, SQL As a basic in tro d u c tio n , when you create an index, you
Server s till performs an INDEX SEEK b u t must perform the can define the percentage o f space on each leaf-level in ­
filte rin g on the fir s t name when i t reads the 99 rows from dex page th a t SQL Server fills w ith data. A nything Less
the clustered index. In th is case, the non-clustered index than 100% represents the remainder th a t SQL Server re­
certa inly helped to narrow the search, b u t SQL Server s till serves fo r future index grow th. I f I specify 90% (often a
needed to "scan" through the 99 rows to fin d a ll the in ­ de fau lt), SQL Server leaves 10% em pty fo r instances when
stances o f "Jeremy". a d d itio n a l data is added to the table. S etting a value too
high (or to o low) in conjunction w ith index fragm enta­
Okay, so w hat happens i f you create a com posite index on tio n can lead to "o p p o rtu n itie s fo r im provem ent."
the Last name and First name?
And Now for Some Basic SQL Syntax
create nonclustered index [ix_LastNameFirstName] Many SQL interview s sta rt w ith making sure the person
on TestPerson ( LastName, FirstName knows the difference between an INNER JOIN, OUTER
JOIN, FULL JOIN, etc. I'm going to go over some exam­
I f you run the query again to retrieve both the last name ples in a moment, even though there are many websites
and the fir s t name, SQL Server performs an INDEX SEEK th a t cover th is . Before I sta rt, here's a Little secret during
and reads ju s t one row in to the Key Lookup to get a ll the SQL assessments: You can p u t a person's knowledge o f
columns fo r th a t single person. You're back in op tim iza­ JOIN types to the te s t by presenting them w ith a scenario
tio n heaven. along the Lines o f th is :

Okay, so a com posite index fu rth e r optim izes the query. "Suppose I have 100 customers in a customer master and
Here's the last question: Suppose I query only on the firs t 100,000 rows in a sales table. You can assume th a t every
name to retrieve a ll the people named Jeremy? W ill SQL sale record is fo r a valid customer. In oth er words, there
Server use the FirstName column from the index and o p ti­ are no orphaned sales rows. I f I do an INNER JOIN be­
mize as w e ll as i t did when I used the last name? Figure tween the rows based on customer ID, how many rows
4 doesn't give us great news. should I expect to get? I f I do a LEFT OUTER JOIN, how
many rows should I expect to get?"
U nfortunately, SQL Server w o n 't perform an INDEX SEEK.
A lthough SQL Server uses the LastNameFirstName index, Yes, th a t's an in te rvie w question flo a tin g o u t there, I kid
i t performs an INDEX SCAN through a ll 19,992 rows. I t you not. The problem is, you d o n 't know i f the person
only finds one " h it " and performs a key Lookup to retrieve is try in g to see w hat oth er questions you m ig h t ask, or
the non-key columns. maybe the person is try in g to see i f the tw o numbers

48 Stages o f Data:The DNA o f a Database Developer, Part 1 codemag.com

(100 and 100,000) w ill be distractors. Ju st Like spelling Okay, let's take the o rig in a l query and turn i t in to a LEFT
bee contestants w ill ask, "Can you use the word in a sen­ OUTER JOIN:
tence," d o n 't be afraid to ask questions about the nature
o f the tab les/ke ys/ro w counts th a t m ig h t be relevant. select Vend.Name,
SUMiTotalDue as VendorTotal
Want to be able to handle questions on different JOIN types? from Purchasing Vendor as Vend
First, practice w ith a database. I f you don't have one, take left outer join
some o f the data in the Microsoft SQL Server AdventureWorks Purchasing PurchaseOrderHeader AS
database or the Contoso retail data. I 'll use AdventureWorks POHon Vend BusinessEntitylD =
for a few examples w ith INNER and OUTER JOIN. POH VendorlD
group by Vend.Name
Suppose I have a Vendor Table w ith 104 vendors. I have a order by VendorTotal desc
sales Table w ith thousands o f sales rows. I have sales fo r
some vendors b u t n o t all. Assume the fo llo w in g : In this case, I can say w ith ce rta in ty (assuming no or-
p h an ed /inva lid foreign keys) th a t SQL Server w ill return
• We have a Vendor table w ith 104 rows. 104 rows. I can say th a t w ith confidence because I'm
• Every PurchaseOrderHeader Vendor ID is a valid va l­ doing a LEFT OUTER JOIN. I'm te llin g SQL Server the fo l­
ue as a Business E ntity ID in the Vendor table. low ing: Give me ALL the rows from the table to the Left o f
• Each TotalDue row in the OrderHeader table is a the LEFT statem ent (the vendor ta b le ), and e ith e r sum the
positive value (i.e ., no negative numbers). vendor dollars or ju s t give me a NULL value i f the vendor
d id n 't have any sales. Given the conditions, I can say w ith
W hat can I say about the number o f rows I'Ll get back high confidence th a t I 'l l get back 104 rows.
from th is query, which w ill generate one row fo r each
Vendor w ith sales, and a summary o f the Order dollars? Now, I'm going to throw the proverbial "curveball." Sup­
pose I w ant a ll the vendors (a ll 104), regardless o f whether
select Vend Name they've had an order. This tim e, I w ant the dollar amount to
SUM TotalDue) as VendorTotal only reflect the sales in 2012. W ill th is give me 104 rows?
from Purchasing Vendor as Vend
join Purchasing PurchaseOrderHeader AS POH select Vend Name,
on Vend BusinessEntitylD = POH VendorlD SUM TotalDue as VendorTotal
group by Vend Name from Purchasing Vendor as Vend
order by VendorTotal desc left outer join
Purchasing PurchaseOrderHeader AS POH
I f someone asks you how many specific rows y o u 'll get on Vend BusinessEntitylD =
back, there's no way to answer th a t. Yes, y o u 'll get back POH VendorlD
one row fo r each vendor th a t had a t least one sale. How­ where (OrderDate BETWEEN
ever, because you don't know exactly how many vendors '1-1-2012' and '12-31-2012' )
DIDN'T have sales, the row count could be as low as 0 ven­ group by Vend.Name
dors or as high as 104 vendors. You're doing an INNER JOIN order by VendorTotal desc
(i.e., basic JOIN) th a t gives you vendors who have sales,
w ill return one row fo r each vendor (because you specified
a GROUP BY), and provides the sum o f the Order Dollars.
If your reply is, "Well, I didn't know
Okay, let's change the query to th is :
that, but I'd never write it that way,"
select Vend Name, take a moment to realize that you
SUM TotalDue) as VendorTotal might inherit code where someone
from Purchasing Vendor as Vend
join Purchasing PurchaseOrderHeader AS POH didn't know, but DID write it that way.
on Vend BusinessEntitylD = POH VendorlD
where (OrderDate
BETWEEN '1-1-2012' and '12-31-2012') Some m ig h t say, "Sure, I 'l l get 104 rows. The dollars could
group by Vend Name be Lower because we're filte rin g on a year, b u t w e 'll s till
order by VendorTotal desc get 104 rows." They m ig h t defend the answer o f 104 rows,
because I've specified a LEFT OUTER JOIN. As one o f the
I'LL ask the same question: What can you say about the Bob contractors said in Office Space, "H old on a second
number o f rows y o u 'll get back and the sum o f the dollar there, professor." You'll only get the vendors who have
amounts? A ll you can say is th a t i t w on't be MORE than an order. (When I run it , I specifically get 79, b u t the
the firs t query, because you've set a filte r condition in the purpose o f th is discussion is to demonstrate th a t I'm no
Where clause to only read the rows w ith an Order Date in longer guaranteed to get a ll o f them ).
2012. I f a ll the orders were in 2012 to begin w ith , I'd gen­
erally expect the numbers to be the same as the firs t query. Here's why. That WHERE clause works a Little * to o * w ell.
I f there were orders in other years, I'd expect either the Even w ith the LEFT OUTER JOIN, the WHERE clause re­
count o f vendors an d /o r the sum o f the dollar amounts to stricts the number o f rows. I've known developers who
be lower. Again, I can't predict the specific number o f rows: were surprised to Learn th is , and I've seen code in produc­
A ll I can say is th a t it's somewhere between 0 and 104. tio n th a t fe ll victim to th is.

codemag.com Stages o f Data:The DNA o f a Database Developer, Part 1 49

 How can you get a ll vendors (in th is case, 104), AND only get good developer w ill surely test ou t the difference and do the
the sales dollars for 2012 (or a NULL). You need to remove necessary research when w riting a query th a t calls for a HAV­
the WHERE from the main query. There are several ways. ING and figure i t out through some tria l and error. Although
Some w ill localize the condition in the LEFT OUTER JOIN, i t m ight seem a b it unfair to expect a person to verbalize the
or (my preference) use a subquery. Here are both solutions: answer to every question, the a b ility to express an answer is
sometimes rooted in the amount o f actual experience. (This
select Vend.Name, is such an im portant distinction th a t I'LL cover i t a second
SUM(TotalDue as VendorTotal tim e in the advanced SQL Server section.)
from Purchasing Vendor as Vend
left outer join That's certainly not a fu ll lis t o f basic SQL Server topics, bu t
Purchasing PurchaseOrderHeader AS POH these are items and nuances th a t I'd expect database de­
on Vend BusinessEntityID = POH VendorlD velopers to know. Before I move on to some advanced SQL
and OrderDate BETWEEN Server topics, I w ant to stop and bring up a point: *M0ST*
'1-1-2012' and '12-31-2012' ) technical interviews are performed by people who can Lis­
group by Vend.Name ten to an answer and judge appropriately. I f you can es­
order by VendorTotal desc tablish a conversational dynamic w ith an interviewer, you
can usually te ll i f the other person is adept a t assessing
select Vend Name your responses. I say "*m o s t*, because there's something
SUM TotalDue) as VendorTotal th a t s till occurs in th is industry, and i t personally in fu ri­
from Purchasing Vendor as Vend ates me, and that's when a "tech screening" is being done
Too Much Theory and left outer join by a person who's given a script. I t happens. Sadly, there's
Not Enough Fresh Air (select VendorlD, TotalDue from no hotline you can call to report such in te lle ctu a l laziness
Purchasing PurchaseOrderHeader (and believe me, I can use stronger words). You can try to
When I was a young
developer, a project manager where (OrderDate BETWEEN call i t o u t in the interview process, bu t obviously you run
fought against me being able '1-1-2012' and '12-31-2012' ) ) POH the risk o f ruining your chances. My only advice is to ju s t
to get my hands on anything on Vend BusinessEntityID = POH VendorlD answer the question as best you can.
other than VERY small test group by Vend Name
rows. He also argued against order by VendorTotal desc
me sitting in meetings with
Some Advanced SQL Server
the business people. He was In Part 2 o f th is series, T il ta lk more about CROSS JOIN Ask ten d iffe re n t SQL developers w hat constitutes "basic
convinced that he and his and SELF-JOIN scenarios. Basically, you use CROSS JOIN SQL" versus "advanced SQL," and y o u 'll get many d iffe re n t
colleague could write the when you w ant to create a Cartesian product o f the table answers. Here's my tw o cents and it's probably controver­
perfect specification, and rows. For instance, say you have 45 employees in a sales sial: A nything th a t a standard 0RM or E ntity Framework
that the team should be able table and a calendar table w ith 52 weeks, and you w ant to generates as SQL code fo r an OLTP application is usually
to write pure and optimized create 2,340 rows, one fo r each em ployee/week com bina­ basic SQL. A nything th a t a data framework can't generate
code without ever having tio n . You use a SELF-JOIN when you need to query a table (or doesn't do w ell, so much so, in fact, th a t i t becomes
a chance to test it against back in to its e lf (i.e ., you query i t as Table A, then jo in a good tra in in g example o f how NOT to w rite som ething)
large or actual data. He even to the same table as Table B, usually because the single is advanced. Yes, th a t's my story and I'm sticking to it .
scoffed when I randomly table has a relationship across colum ns). Again, HL cover
generated a large set of test them more in Part 2, because I w ant to review the to p ic A nything dealing w ith JOIN statem ents on m atching
rows. Reading and studying
o f recursion and common table expressions. keys, WHERE clauses to filte r, and basic aggregation (i.e.,
is important, but it's when
summing the sales dollars across m illions o f sales rows
you get your hands dirty (and
Do you know w hat a UNION does and know the difference and aggregating to one row per Product Brand), a ll o f
crushed by the weight of hard
between UNION and UNION ALL? Here's how I answer th a t th a t falls under basic SQL querying.
lessons) that you become a
question. Suppose you have three d iffe re n t tables (or
data professional.
three query results), w ith the same number o f columns. Most o f the other language features are "n on -ba sic." Okay,
You w ant to append a ll o f them in to one table. I f you I 'l l acknowledge th a t the difference between HAVING and
know fo r absolute certain th a t the three tables do n't WHERE takes Less tim e than explaining the nuances (and
contain duplicated rows (across a ll the colum ns), or you shortcom ings) o f PIVOT because the form er is generally
d o n 't care i f you get duplicate rows, a UNION ALL w ill an "e ith e r/o r" and the la tte r has several parts to it . S till,
give a ll the rows from a ll three tables. I f Table/Result the next ones are some advanced topics.
A had 100 rows, B had 200 rows, and C had 300 rows, a
UNION ALL guarantees a result o f 600 rows. Know the difference between HAVING and WHERE. Here's
the bottom line: You use HAVING to filte r on aggregated
I f there's any risk o f duplicated rows, you should perform a amounts, and you use WHERE when you're directly referring
UNION. A UNION w ill perform a duplicate check based on to columns in a table. Suppose you're retrieving the cus­
all the columns (under the hood, SQL Server performs a SORT tomer ID and the Sum o f Sales Dollars and grouping by the
DISTINCT). This can be expensive for large sets o f data w ith customer. In the query, you only want Customers w ith more
many columns but m ight be necessary. I f someone presents than one m illion dollars in sales. You can use HAVING Sum
the scenario o f three tables w ith the row counts I described, (SalesDollars) > 1000000 after the GROUP BY statement.
and asked what a UNION (no t a UNION ALL) w ill generate for
a fin a l result count, the answer depends on whether any of A lternatively, you can sum the sales dollars by customer
the rows are duplicated across the tables. to a tem porary result set or derived table. When you query
from the tem porary result set or derived table, you can
Do you know the difference between HAVING and WHERE? I use WHERE, because the resulting rows are already ag­
know good developers who stumble on this question. Yes, a gregated down to specific columns.

50 Stages o f Data:The DNA o f a Database Developer, Part 1 codemag.com

Know how th e RAN King functions work. These particu­ gating across m ultiple one-to-m any relationships, where
larly come in handy instead o f try in g to use TOP (N). For the tw o (or more) child tables have no relation to each
instance, i f you w ant to get the to p five selling products other beyond th e ir common relationship w ith the parent.
by state, the RANK functions are preferable to TOP (N).
Also remember th a t there are three flavors o f the general Understand Triggers and Capturing changes. I've w r it­
RANK functions: R0W_NUMBER(), RANK(), and DENSE. ten about th is in prior CODE articles (m ost recently, Janu­
RANK(). The firs t doesn't account fo r ties, the second ary/February 2024), so I d o n 't w ant to repeat details. Most
accounts fo r ties and Leave gaps, and the th ird accounts environm ents need (or require) solutions to track changes
fo r ties and close gaps. Recently I caught an interview er to data. W hether it's w ritin g autom ated scripts fo r tr ig ­
try in g to tric k me in to a TOP N response when the query gers (or ju s t hand-coding trigg ers), or using Change Data
in question called more fo r a RANK fu n ctio n . Capture, or relying on some th ird -p a rty to o l, it's c ritic a l
to know how to log changes and how to report on them .
Select * from (
select PurchaseOrderlD, EmployeelD, VendorlD How do you optim ize queries? You have a query th a t
ShipMethodID, OrderDate, TotalDue suddenly runs slowly and you need to figure o u t why.
It's 2024
rank() over (partition by shipmethodid Here's where there's seldom a single rig h t answer. There
order by TotalDue desc) as RankNum are several diagnostic philosophies here. One o f the and there's
from Purchasing PurchaseOrderHeader ) t th in g s I'Ll do is to h it the low -hanging fr u it firs t. Have no excuse for
where RankNum <= 3 there been new application deployments, did the volume
order by ShipMethodID, ranknum o f data spike astronom ically overnight, can you run SQL
not capturing
Profiler or use the SQL Query store to get a bigger picture, an audit trail
etc. The lis t o f the "low -hanging fr u it" can be very long
of database
Understand how CROSS APPLY and OUTER APPLY work. and five d iffe re n t people can con tribu te 10 uniquely valid
These have been somewhat controversial topics in the ideas, b u t the p o in t is to id e n tify w hether some s ig n ifi­ changes.
SQL Server World. Introduced in SQL Server 2005, they're cant event triggered the drop in performance. You have
great fo r patterns where you need to perform more
gymnastics than a regular JOIN (such as accum ulating, Second, I 'l l review the query, the data, and the execution
Temporal
reading across— or dare I use a COBOL te rm — "Perform plan. Again, I m ig h t be Looking fo r low -hanging fru it, b u t tables (SQL
Varying"). In my last article (CODE Magazine, January/ a t Least it's targeted toward the specific query. Over the
Server 2016),
February 2024), I wrote th a t CROSS APPLY can be used to years, I've become more cognizant th a t a solution using
get cum ulative results. (A dditionally, M icrosoft added op­ views (and views calling views) are n o t unlike blow ing air Change Data
tio n a l clauses in SUM OVER fo r ROWS BOUND PRECEDING in to a balloon: A t some p o in t, you provide one more shot capture (SQL
in SQL Server 2012). There can be a performance h it w ith o f a ir and the balloon pops. My p o in t is th a t adding "one
any o f these features, as they p o te n tia lly need to process more ta b le " or "one more jo in " to a view five d is tin c t
Server 2008),
and aggregate many rows over many itera tion s. I ty p ic a lly tim es doesn't mean th a t the progressive im pact w ill be and you still
use these fo r ove rnigh t reporting jo b s or any instance consistent fo r a ll five. This is som ething I w ant to devote
have triggers
where users accept th a t the results w o n 't be in sta n t. to a future article, b u t fo r rig h t now, keep an eye on solu­
tions th a t have nested views. They sometimes can lead (from the early
Over the years, I've discovered th a t CROSS APPLY has to "to o much air g o t pushed in to the b a llo o n ." In Part days)! Pick one,
some oth er in te restin g uses, such as using a CROSS APPLY 2 o f th is article, I'm going to ta lk more about th is , and
as an alternative to UNPIVOT: how certain execution operators in the execution plan
study it, and
can give you a clue th a t you have an issue. implement it.
create table TestTable
( SaleDate date, DivisionlSales money, I f the problem is indeed "views gone w ild ," what's the
Division2Sales money, solution? Well, " i t depends." I'm n o t a big fan o f views
Division3Sales money) calling views calling views to begin w ith , and so I'Ll try to
delve in to the w orkflow o f the desired query. I f it's some­
insert into testtable values th in g th a t only needs to be produced one or tw o tim es
( '1/1/2023',100 200 300), a day, I 'l l wonder i f a m aterialized snapshot table w ith a
('1/2/2023',200 300 400), crafted stored procedure w ill do the tric k . A lternatively,
('1/3/2023',200 300 01 maybe the c u lp rit is a stored procedure th a t has five que­
ries in it , and maybe the th ird o f the five is the one ta k ­
SELECT SaleDate Division, Sales ing a ll the tim e. U ltim ately, those who can deal w ith bad
FROM testtable performance are the ones who have gone through i t many
CROSS APPLY ( tim es. Quite simply, th is is n 't an easy top ic.
VALUES (' Divisionl' , DivisionlSales ,
('Division2', Division2Sales , Isolation Levels, Snapshot Isolation Levels,
('Division3', Division3Sales ) and Read Uncommitted ("Dirty Reads")
Temp Division, Sales) No one should judge a developer negatively fo r no t know­
WHERE isnull( Sales 0) <> 0 ing about a specific feature. I've known good SQL Server
developers who were unaware o f a feature, or perhaps got
Know when subqueries are required. I talked about i t backwards. Having said th a t, I'm going to go ou t on a
th is in my last CODE Magazine article (January/February lim b and say th a t i f there's one area o f knowledge th a t can
2024), using the seem ingly sim ple example o f m ultiple help you stand o u t as someone w ith good skills, it's Isola­
aggregations and when a subquery is needed. U ltim ately, tion Levels. Odds are (and I emphasize, "odds are") th a t
alm ost every developer runs in to th is situ a tio n o f aggre­ someone who can speak to Isolation Levels, read/w rite

codemag.com Stages o f Data:The DNA o f a Database Developer, Part 1 51

 locks, and the SQL Server snapshot isolation level is likely o f the day) when d irty reads can help, provided those
to show a good command o f other SQL Server fundamen­ instances are well-managed. There's also a th ird risk. This
tals. I'd almost call th is an in te lle ctu a l "gatew ay" feature. one is covered less often in web articles because it's no t
very common, b u t it's s till a risk. Suppose Task A's trans­
In SQL Server, there are five iso la tion levels: Read Un­ action is perform ing inserts such th a t a page s p lit occurs
com m itted (d irty read), READ COMMMITTED (the default in the SQL Server table. I f Task B is using a d irty read, it's
iso la tion level), Repeatable Read, Serializable, and the conceivable th a t Task B could p u ll back duplicate rows
Snapshot Iso la tio n Level (added in SQL Server 2005 and th a t came from the page splits.
i t comes in tw o flavors). I'm going to cover the fir s t tw o
(read uncom m itted and READ COMMMITTED) and the last Okay, so a READ COMMMITTED means th a t users m ight get
one (snapshots) in th is article, and I 'l l cover Repeatable tim eout errors (or w ait longer for the results of queries), and
Read and Serializable in the next article. a d irty read means th a t avoiding those tim eout situations
m ight lead you to reading "b a d /d irty " data. This was a chal­
Any tim e I've ever been asked to ta lk about isolation levels, lenge for a long time. Fortunately, Microsoft SQL Server 2005
I explain them in terms o f "Task A" and "Task B." Both tasks addressed th is w ith one o f the more im portant features in
could be users running an option in an application, or one o f the history o f the database engine: the SNAPSHOT isolation
them could be automated jobs, etc. I f anyone ever wants to level. (Notice how I've skipped past REPEATABLE READ and
test your knowledge o f isolation levels, I highly recommend SERIALIZABLE. I 'll cover them in Part 2 of th is series).
talking in terms o f a "Task A" and "Task B" scenario.
Now I've gotten to the SNAPSHOT Isolation Level. I'm b i­
I 'l l s ta rt w ith the default, which is READ COMMMITTED. ased, but th is is a fantastic feature. This comes in two fla ­
Let's say Task A in itia te s a transaction th a t updates th o u ­ vors, the standard SNAPSHOT (th a t I call a static SNAPSHOT)
sands o f rows in to a table. While th a t transaction is un­ and a READ COMMMITTED SNAPSHOT (th a t I refer to as a "dy­
derway, Task B tries to read some o f those rows. Because namic SNAPSHOT). This gives us the good benefits o f dirty
the default iso la tion level is "READ COMMMITTED" in SQL read and READ COMMMITTED but w ith o u t the downsides.
Server, th a t means Task B w ill be Locked from reading up­
dates from Task A (or could possibly tim e o u t) u n til Task First, to use the standard "s ta tic " snapshot, you need to
A finishes. The specific reason is because Task A's update enable i t in the database, because SQL Server is going to
has a w rite lock on the rows, and th a t prevents Task B use the TempDB database more heavily:
from a tta in in g a shared lock to read the com m itted ver­
sion o f the rows. Task B needs to w a it fo r Task A to f in ­ -- Code to alter a database to suppport
ish and com m it the transaction (or fo r Task A to rollback -- static snapshots
because o f some error in Task A). ALTER DATABASE AdventureWorks2014
SET ALLOW_SNAPSHOT_ISOLATION ON
There can be problems here, and discussions are not w ith ou t
controversy. Task B m ight simply need to get an "approxima­ Here's the scenario fo r a s ta tic snapshot:
tio n " o f data. In other words, i t doesn't m atter much i f Task
A commits or rolls back; it's small in the grand scheme o f • Customer XYZ has a cre d it rating o f "Fair".
what Task B wants to query. What Task B can do is set the • Task A starts a transaction and updates the custom ­
isolation level to a read uncommitted (d irty read). The ben­ er's cre d it rating to "Good". This transaction could
e fit is th a t Task B w on't run a risk o f tim eout w aiting for Task take several seconds or even m inutes to com plete i f
A to finish, and a d irty read is sometimes a b it faster than it's a long and intensive jo b
a normal READ COMMMITTED, because SQL Server isn 't even • You've seen th a t Task B w ill have to w a it (or m ig ht
attem pting to put a shared lock on the rows. even tim e o u t) i f you use the de fau lt READ COM­
MMITTED. You've also seen th a t Task B returns a
Sounds great? Well, d irty reads bring some real risks, and value o f "G ood" i f i t uses a READ UNCOMMITTED,
here are three o f them . which is n 't "g o o d " i f Task A rolls back.
• Suppose th a t Task B starts its own read session/read
First, suppose task A's transaction is updating both a transaction and sim ply wants to get the last com m it­
header table and tw o child d e ta il tables. Task B performs ted version ("F air"). Task B can in itia te a transaction,
a d irty read between the update o f the header table and set the ISOLATION LEVEL to "SNAPSHOT", and then
the child tables. Task B sees the new header row b u t query the table: Task B's query returns "Fair". I t w on't
doesn't y e t see the related d e ta il rows. In other words, give you a d irty read and i t w on't give you a tim eout.
Task B returns an incom plete picture o f w hat Task A's
transaction intended to w rite. That's n o t good!
Many DBAs refer to snapshot

I
Second, suppose Task A starts and updates one or more of
the tables, but performs a rollback at the end (because of
some post-validation error, etc.). Suppose Task B read the
isolation as versioning.
data using READ UNCOMMITTED ju s t before the rollback. Task
B w ill be returning data th a t never officially saw the lig h t
o f day because Task A rolled i t back. That's also not good! Sounds great, doesn't it? Overall, i t is, although there's
one dow nfall. Suppose Task A com m its its transaction,
These tw o alone should provide caution to developers who b u t Task B is s till in the m iddle o f its READ session o f
use READ UNCOMMITTED. Again, there can be specific in ­ SNAPSHOT. I f Task B queries the row again (in the same
stances (often tie d to w orkflow th ro u g h o u t the course read session), i t continues to return "F air" because i t

52 Stages o f Data:The DNA o f a Database Developer, Part 1 codemag.com

continues to read from the last com m itted version AT "The Gang Of Four", Erich Gamma, Richard Helm, John
THE TIME its own READ session started. (This is why VLissides, and Ralph Johnson) came up w ith some great
many DBAs refer to snapshot iso la tion as versioning). "s h o rt b u t sweet" names fo r patterns. Well, I'm n o t ta l­
ented enough to come up w ith "sh o rt b u t sweet term s,"
Here's w hat M icrosoft did to handle th a t. In my op in io n, b u t I have a name fo r th is pattern. I call i t the " s tu ff
th is ranks as one o f the fin e s t achievements in the his­ the X number o f variable business values in to a single
to ry o f the SQL Server Database engine. They added an comma-separated lis t, and then jam th a t as a single new
o p tio n a l clause to the SNAPSHOT iso la tion Level. By per­ column in the report. (Side note: I'd never succeed in
form ing the fo llo w in g s e ttin g , you are guaranteed th a t product branding.)
EVERY default READ COMMMITTED w ill give you the most
recently com m itted version. I've had business users who were happy w ith th is result,
and IT managers who breathed a sigh o f relie f th a t there
- - Code to alter a database to suppport was a simple solution th a t w ouldn't disturb the expected
- - READ COMMMITTED SNAPSHOT row count.
- - This will configure SQL Server to read
- - verions from TempDB Okay, so how can you do this? Prior to SQL Server 2019, a
common approach was to use the "FOR XML" statem ent.
ALTER DATABASE AdventureWorks2014 In SQL Server 2019, M icrosoft added a new STRING_AGG
set read_commi tted snapshot on fu n ctio n to read over a set o f rows and "aggregate" values
in to a comma-separated string . Take a Look a t STRING_
Go back to the previous example. I f Task B reads the row AGG, which works very nicely. Also, Look a t STRING_SPLIT,
a second tim e, i t picks up the change from Task A. Again, a Long overdue feature to easily do the reverse: convert a
it's because o f the dynamic nature o f READ COMMMITTED comma-separates string o f value in to a result set.
SNAPSHOT.
Second, th is is one th a t in itia lly seems Like a simple jo b
You d o n 't even need to s ta rt read sessions w ith a SNAP­ fo r MAX and GROUP BY b u t winds up being a b it more in ­
SHOT iso la tio n . By tu rn in g on READ_COMMITTED_SNAP- volved. Because I've established m yself as a long-winded
SHOT in the database, every single READ COMMMITTED branding author, I'd Like to refer to th is pattern as the
(th e d efau lt level) autom atically pulls the most recently " I th o u g h t I only needed a GROUP BY and MAX, b u t now
com m itted version o f the row rig h t a t th a t tim e. I need to go back and p u ll one oth er column th a t's no t
dire ctly part o f the aggregation." (I'd fa il m arketing.)
Does th a t mean th a t the READ COMMMITTED SNAPSHOT
w ill work wonders rig h t away? I t m ight, b u t there's one Suppose you have a query th a t in itia lly shows the single
catch. As SQL Server is using TempDB as a version store, highest transaction by customer. I t could be som ething
DBAs w ill Likely need to step in to deal w ith the manage­ as sim ple as th is :
m ent o f TempDB.
SELECT CustomerlD, Max(SaleAmount)
From Sales
Now for Something a Little Bit Different GROUP BY CustomerlD
Here are tw o more SQL Server tricks and approaches th a t
experienced database developers have up th e ir sleeves. But now you're asked to show the Sales Person associated
w ith th a t sale. You can try a ll day Long, b u t you w o n 't be
First, suppose you have a report th a t shows one row per able to w rite th is as a single query. You'Ll need a subquery
product w ith the sum o f sales. The company sells the o f some type.
product in d iffe re n t sizes, b u t the report only shows the
summary o f sizes. A dditionally, you could have a tie w ith in a customer.
Maybe the customer bought som ething from Sales Per­
Now suppose the business has an urgent request. A t any son A fo r $100 and som ething else from Sales Person B
one tim e, maybe tw o or three o f the sizes need to be fo r $100. There are tw o sales transactions fo r the same
replenished (which you'd p u ll from a query). You w ant to am ount w ith a d iffe re n t Sales Person.
show the List o f sizes, except the report row granularity
is fixed and you can't add a d d itio n a l row d e fin itio n s to Here's another situ a tio n when sp o ttin g th is pattern (and
the report. Also, because there m ig h t be 20 sizes, you w ritin g a correct query) can sometimes in dicate a devel­
can't add 20 columns o u t to the rig h t o f the rest o f the oper's level o f experience.
data, w ith a checkbox fo r the sizes you need to replenish
because th a t would take up to o much space. ;with tempcte as
( SELECT CustomerlD,
But wait! There's something you can do. After talking to the max(SaleAmount) as MaxSale
users, they ju s t need to know the X number o f sizes, nothing from Sales Group by CustomerlD)
else. It's almost like seeing a to o ltip on a webpage, except
this is a generated report. Maybe all you need to do is ju s t Select tempcte.*, OriginalTable.SalesPersonlD,
add one more column to the report, called "Sizes to Replen­ OriginalTable.SalelD, OriginalTable.SaleDate
ish", and show a comma-separated List o f the sizes? From Tempcte
Join dbo.Sales OriginalTable
I f you've ever read the famous "Design Patterns" book, on Tempcte.ID = OriginalTable.ID and
y o u 'll know th a t the authors (a ffe ctio n a te ly known as tempcte.MaxSale = OriginalTable.Saleamount

codemag.com Stages o f Data:The DNA o f a Database Developer, Part 1 53

 There are multiple ways to write this. Some might use • Rows in the Legacy system with cost values th a t the
the Common Table Expression as an in-line subquery, and target system could not process (dirty data th a t had
some might use CROSS APPLY, etc. This point is th a t when never been validated)
you aggregate across many rows to come up with a SUM • Multiple rows in the target system th a t were marked
or a MAX or whatever, you might need to bring along as active, when only one row should have been
some additional columns for the ride. (Aha! Maybe I can marked as active, based on the granularity
call this the "Tag-along" aggregation pattern!)
Before I go on, I want to talk about th a t Last word: granu­
Finally, I want to bring up a pattern th a t I've only had to larity. I t might sound pedantic to invoke th a t term during
deal with twice (to my recollection). I t was a b it unusual, a meeting about fixing data, but as i t turns out, three
and very humbling. I had to write a query th a t summa­ different functional areas in the company had different
rized process duration, and mistakenly thought th a t a few opinions about what constituted "granularity of an ac­
subqueries and aggregations would do the trick. I t turned tive row." The problem is th a t some business users aren't
into a hair-pulling afternoon, although I benefitted from going to think in the most precise terms about this (nor
it. I came up with a working solution, and then learned should they be expected to ). They'll simply look at three
later th a t another SQL Server author (Itzik Ben-Gan) had candidate rows on a screen, point to the second, and
documented the pattern, called "gaps and islands." Be­ say, "THAT ONE! That's the active row, and the others
cause I already presented the code in a prior CODE Maga­ shouldn't be, because of this and this and this."
zine article (January/February 2018), I'm not going to re­
paste the code here. I f you want to see a more advanced And that, folks, is how you come one step closer to defining
On the Subject of example of where a developer can mistakenly oversimplify (in this case) what constitutes the definition of an active row.
Starting Out a problem, I documented my entire thought process in
the January/February 2018 issue of CODE Magazine ("A Okay, back to the original story, one of the first things we
There's no single roadmap
SQL Programming Puzzle: You never stop learning"). provided to management was a summary of what percent­
in learning a discipline (in
this case, being a database age of bad rows fe ll into each category. We had roughly
6,000 rows identified as "bad" with 20% falling into Cat­
developer). As people often Data Profiling: I've Seen Fire (Audits) egory A, 50% into Category B (and then a sub-breakdown
say, "It's a journey." Just make
sure "journey" is a verb as well and I've Seen Rain (More Audits) of Category B), 10% into Category C, etc.
as a noun. I 'll admit, I have a b it of a wise-guy side to me, and it
nearly came out in a recent conversation. Someone asked The queries were a series of COUNT and EXISTS
i f I'd done data profiling and the proverbial little devil
on my left shoulder wanted to say, "Well, I don't know DECLARE @CategoryACount int , @CategoryBCount int
th a t specific term, but in my last project, I had to take a
large number of rows from a bad OLTP system and search SET @CategoryACount = (select count(*) from
for patterns." CostDetaiIsLegacy outside
where MarkedArchive = true
Obviously, I didn't make th a t joke, but here's what I DID And exists
say: You can't spell "accountability" w ithout "count." (select 1 from CostProduction inside
To me, accountability isn't about "who can we blame if where outside.<Keyl> = inside.<Keyl> and
something goes wrong?" Accountability means "what can outside.<Key2> = inside.<Key2> and
we count on" or, in the world of databases, "all rows are Inside.ConditionForActive = true
present and accounted for in some manner."

For example, I worked on a project where we had some SET ©CategoryBCount = (select count(*) from
pretty significant cost discrepancies between two sys­ CostDetaiIsLegacy outside
tems. We knew the issues stemmed from code between where MarkedArchive = true
the two systems th a t needed to be refactored. Before we And exists (select 1 from CostProduction inside
could dive into that, we had to come up with a plan where outside.<Keyl> = inside.<Keyl> And
RIGHT AWAY to fix the data. Of course, you can't fix a outside.<Key2> = inside.<Key2> and
problem (or in this case, a myriad of problems) without inside.ConditionForActive = true
identifying all the issues, and th a t was the first order
of business: identifying all the different ways data had Additionally, management might provide an error factor:
gone bad. Maybe i f cost rates differ by Less than two cents (round­
ing errors, bad math approaches involving integer divi­
Without going into specifics, we found four different sion, etc.), and *maybe* they'll elect to tackle th a t later.
scenarios. Of those four, two of them had sub-scenarios. Once I saw a manager flip out when they saw the top
Some of these were simple and "low-hanging fru it" and of the list of discrepancies sorted by variance and the
some were more complicated. Here were some of them: overall row count. They thought th a t because the top 10
rows were o ff by a large percentage and we had thousands
• Rows in the legacy system marked as deleted/ar- of rows, th a t we had a disaster. I t turns out th a t after
chived, but s till in the production system row 20, the variances dropped to pennies, with a slew of
• Rows in the legacy system marked as deleted, but numbers o ff by ju s t a small amount. So even within your
should not have been categories, check the deviation among the rows.
• Rows in the legacy system with multiple cost com­
ponents, where the target system only ever recog­ I'm not going to devote two pages of code to the specif­
nized the first component ics ( it'll ju s t give me flashbacks and nightmares anyway).

54 Stages o f Data:The DNA o f a Database Developer, Part 1 codemag.com

Hopefully you get the idea: You need to identify all the and Y, and what are the differences between X and Y?"
anomaly conditions before you can fix them. Addition­ Person B might very well have worked on projects involv­
ally, management might decide to go over a particular ing X and Y, but might stumble on a definition, and finally
scenario first, because it's the most egregious, or easiest respond, "Well, I can't give a textbook definition, but I've
to fix, etc. worked on both and know they're different, and I'm sure
I can easily Look them up on Google." Person A might ac­
Obviously, this won't fix any data, but i t gives you a good cept th a t when push comes to shove, Person B "probably"
structured starting point for a game plan, as opposed to understands enough.
playing the proverbial whack-a-mole when you see bad
data. This is what I mean by data accountability: what When developers face questions about the difference
rows you can "count on" as being valid, which ones you between normalization and denormalization and try to
can't, and why. Trust me, the auditors are trained to think describe some common terms in data warehousing, they
in these terms (and others). sometimes can fa ll a little short, even i f they've success­
fully worked on projects. Yes, I realize th a t memorizing
standard answers might not be the way to go, b ut....
Always Know the Product/Language
Shortcomings Let's take normalization and denormalization first. The
I was on an interview team where I asked a question that goal of normalized databases is to reduce/eliminate re­
some liked and some wondered what planet I came from. dundancy, to ensure data integrity, and to optimize get­
I t was for an ETL position where SSIS and T-SQL would be ting data into the database as efficiently as possible.
used heavily, and so I asked, "What three things about You might have to write queries to jo in data across many
SSIS annoy you the most th a t you wish worked differ­ tables, but a SQL Server developer shouldn't fear that
ently?" prospect. Typically, application developers are the only
individuals who'll be accessing normalized tables with
I wasn't so much interested in the specific answers as SQL code.
much as I was interested in their though process and
product experiences. At the risk of making an abso­ Denormalized databases are a different breed: Power us­
lute statement, it's unlikely th a t someone has a strong ers and "non-developers" might be more likely to read
amount of prior experience with SSIS and has been a them because data can be redundant in denormalized
happy camper about the product all the time. I Love SSIS tables. One of the major goals of denormalized databases
and I've debated with developers who refused to use it, is to optimize getting data OUT of the database as quickly
but th a t doesn't mean I haven't (out of frustration) come as possible. Fewer complex JOIN statements are needed,
up with alternate definitions for what the letters SSIS which can appeal to power users who know how to use
stand for! The SSIS Metadata manager in the data flow reporting tools but want to query the data with minimal
has gone from frustrating to barely acceptable over the effort.
years. SSIS's handling of package-level variables at the
script Level makes me think of two people talking with Finally, something you can use to impress your friends
plastic cups and a string. I have other pet peeves. Again, at the dinner table: Some data warehouse initiatives
I love SSIS, but the product has weak spots, and I'm cu­ (especially ones with a large number of disparate data
rious to know what a developer thinks is a weak spot sources) shape data into normalized structures FIRST
because i t usually means they've cut their teeth with it. as part of an overall ETL effort, before turning around
and flattening them into denormalized structures. For
When Microsoft first implemented the PIVOT in T-SQL developers who've only ever worked in one type of data­
2005, I was mildly excited. I had done many queries in ­ base, this concept might in itia lly seem like overkill, but
volving aging brackets, and Looked forward to converting it's an invaluable approach for making sure data in the
some code I was writing to use PIVOT. The story's end­ data warehouse is clean.
ing wasn't quite as happy as I expected. I t streamlines
some things, but also introduces other "gotchas" to be Yes, the capabilities of certain BI and reporting tools, and
aware of (dynamic lists, multiple columns, and the need even advancements in database features, have created
to PIVOT on one pre-filtered/pre-defined result set, as some grey areas, but these two concepts s till breathe on
opposed to mixing PIVOT with other T-SQL constructs and their own today, and they're not going away.
with an irregular result sets). I was once asked in an
interview what new Language feature gave me the most The topic of data warehouses is a b it of a different story.
frustration. I chose PIVOT: not because I didn't under­ There are developers who can cleanly describe the differ­
stand it, but because I'd had enough practical experience ences between normalized and denormalized databases
th a t I *sometimes* wondered i f the pre-SQL Server 2005 but might admit to a shortfall of knowledge on data ware­
approach was better. I'm reasonably confident th a t the housing concepts.
interviewer would have accepted different answers, so
long as I said something th a t demonstrated I'd gotten my I f you plan to work in the data warehousing area, there's
hands dirty with it. one book I absolutely recommend you read: "The Data
Warehouse Toolkit: The Complete Guide to Dimensional
Modeling" by Ralph Kimball and Margy Ross. There are
Normalization, Denormalization, multiple editions of this book. Even i f you can only get
and Warehousing Basics access to the first edition, it's s till worth reading. I real­
There's a classic argument th a t goes along the Lines of ize th a t i t almost sounds cultish to recommend a book so
this: Person A w ill ask, "What's a short definition for X strongly, but the book actively engages in different busi-

codemag.com Stages o f Data:The DNA o f a Database Developer, Part 1 55

 ness cases where Kim ball and Ross use repeatable p a t­ w ith an answer sheet who are unable to determ ine i f a
terns to handle d iffe re n t scenarios. I t s n o t ju s t a book to pa rticular response aligns w ith a predefined answer.
quote, it's also a book w ith approaches to live by.
Okay, Let's go back to fa c t tables. The scenario I described
(a sales record) is the most common type o f fa c t table:
Next time someone says, "Why a transactional fa c t table. I can aggregate a ll the sales
dollars (or aggregate by dim ensional groupings) and th a t
measure w ill mean som ething. You refer to the measures
should I study the Kimball in a transactional fa ct table as fu lly additive.
methodology? Isn't that

I
Certain businesses w ant to take a picture o f the sum o f
antiquated?" here is a proper reply.
transactions over some period. Im agine a bank takin g
"Do you think that all databases a picture o f a ll a person's charges and a ll th e ir depos­
built on Kimball fundamentals its during the month and coming up w ith a balance. A
company m ight, as part o f a m onth ending process, read
have suddenly disappeared?" across a ll the bank transactions and w rite o u t a periodic
Knowing how to implement Kimball snapshot table th a t holds the account ID, m onth ID, sum
o f charges, some o f deposits, and the balance. Let's say
ideas will be relevant for a LONG
they do th a t fo r three months and so the snapshot fa ct
time. Its not a technology, table holds three rows. Can you sum the numbers fo r the
it's not a language: It's a collection charges and deposits in to som ething m eaningful? In th is
case, yes. Can you sum the balance fo r each o f the three
of approaches, and an outstanding months? Well, much as you m ig h t Like th a t, the balance
one that that. is n 't cum ulative. Unlike the fir s t tw o measures, the bal­
ance is n 't fu lly additive. Yes, you could take an average
o f it , which means th a t measure is sem i-additive, and
n o t cum ulative. Periodic snapshot fact tables are not
There are many d iffe re n t data warehousing concepts, bu t uncommon.
here are a few you should know:
There's also a th ird type o f fa c t table, usually in vo lvin g an
First, w hat are Fact Tables and Dimension Tables? a c tiv ity where you're n o t Looking to sum up numbers (like
Loosely speaking, Fact Tables represent a c tiv ity /b u s i- sales or number o f item s produced) b u t merely counting
ness processes. They could be sales, returns, transactions, the number o f tim es som ething happened. For instance,
m anufacturing production and quality, or even survey let's say I teach tw o classes a day. Yesterday, 28 o u t o f
evaluations: SOMETHING happened, and you w ant to ana­ 30 attended Class A, and 18 o u t o f 19 attended Class
lyze those som ethings over tim e. They could occur five B. The next day, the numbers are 29 o u t o f 30, and 17
m illio n tim es a day or ju s t 100 tim es a week. Last n ig h t, o u t o f 19, respectively. You m ig h t have a Factless fa ct
I bought some new a rt supplies fo r my daughter— th a t table th a t records no thin g more than the number o f tim es
transaction m ig h t very w e ll be s ittin g in a fa c t table. More som ething happened (attendance fo r a pa rticular class
on th a t in a m inute. by a pa rticular in stru cto r). A ll you really get o u t o f the
Factless fa c t table is the count, although th a t count over
Dimension tables represent the business e n titie s th a t are tim e (or average) m ig h t have analytic value.
associated w ith the a c tiv ity m entioned above. I bought
the supplies a t a certain tim e o f the evening from a store Okay, I'm going to bring up one more to p ic th a t's likely
in a pa rticular sales d is tric t th a t belongs to a certain to come up during an interview : slowly-changing dim en­
sales region. The products were specific a rt supplies made sions. Here's how I describe it : suppose a product was
by a certain company and came in certain packaging. introduced in 2022 and sold in a ll o f 2022 fo r a manu­
facturer price o f $50. On 1 /1 /2 0 2 3 , the price increased
Here's the single most im p o rta n t word in data warehous­ to $52. Then on 7 /1 5 /2 0 2 3 , the price increased to $53.
in g: the word "b y ." Sales departments w ant to see the
rollup o f sales dollars "b y " city, "b y " customer type, "b y " Here's the big question: Do you care about tracking sales
da te/m on th or even "b y " tim e o f day, "b y " product manu­ during the tim e when the product was $50, versus $52,
facturer and product type. So those core id e n tifie rs th a t versus $53?
were part o f the sale (the exact d a te /tim e , the customer
ID, the product UPC, the store ID and maybe even sales Here's another example. From the tim e I started paying
register number) belong to larger groups. taxes (i.e ., age 18, fir s t jo b ), I've Lived a t 10 d iffe re n t
addresses (kind o f scary to th in k about it ) . Let's say some
I f the product was a seasonal item , sales m ig h t w ant to governm ent agency tracks in fo rm atio n about me. Do they
track sales across years by season (seasonality), where care i f was a resident in one area fo r X years/m onths and
the actual dates m ig h t vary a b it. ( I Like to use the classic another area fo r a d iffe re n t am ount o f tim e?
example o f tracking sales o f fish foods during Lent each
year, where Lent can be d iffe re n t weeks each year). When you care about aggregating fa ct data n o t only by
an a ttrib u te (a product ID or a Social Security Number),
Now, you can look up fa c t tables and dimension tables b u t by changes to th a t a ttrib u te th a t have occurred over
and fin d several equally valid d e fin itio n s. This is one o f tim e, you refer to th a t as a Type 2 Slowly Changing Di­
the reasons I detest unqualified people doing interview s mension.

56 Stages o f Data:The DNA o f a Database Developer, Part 1 codemag.com

Now you're wondering, "Okay, what's a Type 1 Slowly some very irregular production history, believing th a t I'd
Changing Dimension?" It's sim ply when some a ttrib u te somehow concocted a magic form ula to suddenly show
fo r an e n tity changes many tim es and there's no analytic w hat happened on a day three months ago. I t wasn't
need to aggregate fa c t data over tim e by each change. magic, i t was basic data warehousing concepts. A t risk
o f re p e titio n , and a t risk o f our famed e d ito r strikin g
There are challenges associated w ith Type 2 Slowly Chang­ th is sentence because it's o v e rk ill (sm ile), I can't stress
ing Dimensions: Late-arriving fact data and Late-arriving enough the value o f reading the Ralph Kim ball m ethod­
dimensional changes. On the former, suppose you receive ologies in his Data Warehousing books.
sales data today (Late A p ril 2024), bu t i t represents trans­
actions th a t occurred in January 2024. When you w rite ou t Ever see those TV commercials where tw o people are debat­
the fact data, you need to make sure the fa ct row points ing about whether something happened the day before?
to the proper VERSION o f th a t dimension a t th a t tim e. On Suddenly someone shows up w ith in sta n t replay equipm ent
the Latter (and th is can be trickie r): suppose you receive to show w hat happened. You want to be the person who
inform ation today th a t a product's price changed back on can show up w ith the necessary replay equipment.
11/23/2023? You m ight be tem pted to say, "Okay, w e'll go
back and update the dimensional pointers for those fact
sales transactions th a t occurred after 11 /23 /2 02 3 to p o in t Other Miscellaneous Things
to the correct version o f the product's change." Well, th a t Think globally: Your applications aren't the only ones th a t
depends on the database policy regarding "u p d a tin g " fact m ig h t touch the data you manage. When you're Looking
tables. In some environments, fa ct rows are INSERT ONLY— a t a u d it tr a il requirements, you can't assume th a t your
you CANNOT update a fa ct row. In th is instance, you'd fix processes are the ONLY ONES th a t w ill touch data. Once I
the X number o f fa ct rows th a t would have been associ­ g o t in to a spirited argum ent w ith a SQL Server author th a t
ated w ith the new version o f the product change (had you I highly respected. In the argument, the author stated
received i t in a tim e ly manner), INSERT new rows w ith re­ th a t the new OUTPUT INTO clause th a t M icrosoft added
versing sales amounts, and then INSERT the rows the way to SQL Server 2005 meant th a t triggers and oth er a u d it
you would have in the firs t place. Yes, th is happens! tr a il features would go away. He defended his position
by saying he'd never work fo r a company th a t d id n 't pipe
ALL th e ir insertions through stored procedures. I to ld him
And Speaking of Snapshots... th a t i t ju s t w asn't practical to th in k th a t way, th a t other
A t some p o in t, many businesses need to see w hat data jo b s m ig ht access and m odify data.
looked like based on a p o in t in tim e. This can take a
variety o f forms. Suppose you report on daily production
every m orning a t 5AM. You m ig h t w ant to see w hat a
product looked like between January 10 and January 11 In the database world, there are
(possibly because o f some s ig n ific a n t event). Here's an­
other one: maybe you need to see w hat a customer profile
a few RED rules (rules you always
looked Like back on February 3. Finally, suppose you w ant follow) and lots of BLUE rules (rules
to show the trend o f in ven to ry on the firs t day o f the
you try to follow but might need
m onth, the last day o f the m onth, and any m onthly aver­
ages, over the course o f a year. to bend). At the end of the day,
the pragmatists usually prevail—
I f i t sounds Like I'm p a rtly repeating myself (th a t is, I'm
m entioning some o f the scenarios from the data ware­
so long as they also have a plan
housing section), you're correct. Snapshot tables, au dit for tomorrow and the next day.
tr a il tables, and slowly changing dimensions, they a ll have
some overlap. Even i f you d o n 't work in a full-fled ge d
data warehousing team , your a b ility to solve some o f
these challenges can increase your value. Some people Here's another to p ic : using PowerShell. I 'l l freely adm it,
have th is impression o f data warehouses as a bunch o f old I d o n 't ju m p w ith jo y when I use PowerShell. However, i t
backup tapes from data 30 years ago th a t's kept o ffs ite in can be an invaluable to o l fo r "p iecing to g e th e r" d iffe re n t
some dusty storage area. I th o u g h t th a t way a t the begin­ processes, especially database processes.
ning o f my career, and I could n o t have been more wrong.
Suppose you need to run a standard ETL data jo b , exe­
Database professionals have d iffe re n t opinions about cute some custom API o u t in the operating system, and
the structure o f such tables. On the one extreme, some then run a fin a l database job? You'd like to use SQL Server
m ig h t sim ply use change data capture to log every single Agent, as you could easily add Steps 1 and 3 as jo b steps,
change, and then use queries to reconstruct history at and then place the PowerShell script in between the steps.
any one p o in t in tim e. Granted, th a t could be a lo t o f SQL
code, b u t i t can be done. Some create tem poral tables Here are some ways I've used PowerShell:
(or use the tem poral table feature added in SQL Server
2016) to possibly reduce the am ount o f code needed to • To deploy reports to an SSRS or Power BI report
reconstruct history. Some create periodic snapshot tables server area under a d iffe re n t service account
where the periods represent specific established business • When I had a specific custom process where I need­
tim elines (beginning o f m onth, end o f m onth, average ed to tap in to AD0.NET in between jo b steps, and
during the m onth, etc.). On one occasion, a c lie n t was SSIS and .NET modules were n o t a deploym ent op­
stunned th a t I was able to reconstruct a tim e lin e to show tio n .

codemag.com Stages o f Data:The DNA o f a Database Developer, Part 1 57

 Recommended Reading and I tr ie d t o fo c u s m ore on core s k ills in th is a rtic le . A nd
t r u s t me, a m o n th a fte r th is issue is p rin te d , I ' l l say to
Experimenting m y s e lf, "R a ts, I fo r g o t t o m e n tio n som e appro ach t h a t
There are m any fa n ta s tic SQL a u th o rs o u t th e re , such as c o u ld have made th is a rtic le b e tte r ." For t h a t reason, I'm
I t z ik Ben-G an, Greg Low, B re n t Ozar. They have c o n te n t s p lit t in g th is a rtic le in t o m u ltip le p a rts.
on m u ltip le w e b s ite s . T h a t's j u s t th re e , and th e re are
m any o th e r g re a t ones as w e ll. B u t here's so m e th in g I do w a n t to say. Those w ho fo llo w
sports m ig h t re ca ll a pla ye r w ho was c ritic iz e d fo r n o t g iv ­
I'v e w r itte n on SQL Server, T-SQL, and Data W arehousing in g f u ll e ffo rt d u rin g p ractice sessions, and reacted d u rin g
c o n c e p ts in m any CODE M agazine a rtic le s . You can go to a press conference by re p e a te d ly sayin g, "C 'm on, we're ta lk ­
h ttp s ://c o d e m a g .c o m /P e o p le /B io /K e v in .G o ff and see a in g a b o u t practice. N ot a game, b u t p ra c tic e ." Nearly a ll o f
lis tin g o f a ll m y a rtic le s . us can th in k back to some tim e in o u r Lives w hen we g re a tly
im p ro ve d o u r s k ills in some area. I'm b e ttin g t h a t more
Here are ones I'd Like to c a ll o u t in p a rtic u la r: focus and more p ra ctice and more s tu d y in g played a role.

• Two T-SQL a rtic le s R eading SQL S erver books and b lo g s is g re a t, b u t w h a t's

• h t t p s : / / c o d e m a g . c o m / A r t ic le / 2 4 0 1 0 5 1 / S t a g - even g re a te r is ta k in g som e o f th o s e s k ills and tr y in g
e s - o f-D a ta -S o m e -B a s ic -S Q L -S e rv e r-P a tte rn s - th e m o u t on y o u r ow n databases. M ic ro s o ft has A d ve n ­
a n d -P ra ctic e s tureW orks and C ontoso R e ta il dem o databases. W hen I
• h ttp s ://c o d e m a g .c o m /A rtic le /1 8 0 1 0 5 1 /A -S Q L - w ro te a rtic le s on COVID d a ta , I fo u n d m any Excel/CSV

SPONSORED SIDEBAR P ro g ra m m in g -P u zz le -Y o u -N e v e r-S to p -L e a rn in g file s w ith s ta tis tic s . Yes, i t to o k som e w o rk to assem ble
• A Power B I a rtic le : th o s e in t o m e a n in g fu l databases, b u t i t was w o rth i t . I f
Ready to • Stages o f D ata: COVID D ata, S um m ary Dash­ y o u 're s ta r tin g o u t a t a new jo b , o r even a p p ly in g fo r a
boards, and Source Data Tips ( c o d e m a g .c o m ) new jo b , y o u w a n t p e o p le to w a tc h y o u do s o m e th in g and
Modernize a
• Four a rtic le s on SQL S erver re p o rtin g S ervices: say, "W ow, t h a t person has o b v io u s ly done th is b e fo re ."
Legacy App? • h ttp s ://c o d e m a g .c o m /A r tic le /1 8 0 5 0 5 1 /
R e fa c to rin g -a -R e p o rtin g -S e rv ic e s -R e p o rt-w ith -
Need advice on migrating
yesterday's legacy Som e-SQ L-M agic
Summary: What I Almost Called
applications to today's • h t t p s : / / c o d e m a g . c o m / A r t ic le / 1 7 1 1 0 6 1 / S Q L - This Article
modern platforms? Take S erver- R e p o rti n g -S e rvi ces- Ei g h t - Po w e r-T i ps I'v e been w o rk in g on th is a rtic le fo r o v e r fo u r m o n th s. As
advantage of CODE • h t t p s : / / c o d e m a g . c o m / A r t ic le / 1 7 0 5 0 6 1 / S Q L - m o s t a u th o rs can a tte s t, w h a t y o u s ta r t w ith and w h a t
Consulting's years of S e rve r-R e p o rtin g -S e rvice s-S e ve n -P o w e r-T ip s y o u fin is h w ith can be d iffe r e n t th in g s . As I lo o k back
experience and contact • h t t p s : / / c o d e m a g . c o m / A r t ic le / 1 6 0 5 1 1 1 / S Q L - o ve r th is , th e c o n te n t it s e lf d id n 't change m uch, b u t th e
us today to schedule a S erver- R e p o rti n g -S e rvi ces- Ei g h t - Po w e r-T i ps reasons I w ro te i t e v o lve d . As I m e n tio n e d e a rlie r, I'v e
FREE consulting call to seen m any L in k e d ln q u e s tio n s w here s o m e th in g Like th is
discuss your options. I'v e cre a te d d a ta p ro je c ts and dashbo ard pages fro m p e r­ w o u ld be h e lp fu l. I also w ro te th is because I w a n te d to
s o n a l d a ta fo r e v e ry th in g fro m m y w e e k ly h e a lth s ta ts to share w h a t th in g s I'v e seen m any tim e s . I'LL ne ve r c la im to
No strings. No
p e rs o n a l fin a n c e s . The m ore y o u p ra c tic e , th e b e tte r! have a ll th e answ ers on w h a t makes a go od datab ase de­
commitment.
ve lo p e r, b u t I'v e in s tru c te d p e o p le a t a te c h s c h o o l w hose
It's g re a t t o read and absorb in fo rm a tio n fro m w e b s ite s m issio n was to h e lp p e o p le g e t jo b s (o r g e t b e tte r jo b s )
For more information,
w w w .c o d e m a g .c o m / and books. Yes, so m e tim e s it 's because y o u 're tr y in g to and I'v e m e n to re d o th e r d e ve lo p e rs. I alw ays w a n te d to
c o n s u ltin g or email us at so lve a s p e c ific p ro b le m a t w o rk , so y o u a lre a d y kno w ta k e an in v e n to ry o f w h a t fu n d a m e n ta ls I t h in k o th e rs
in fo @ c o d e m a g .c o m . y o u 're g e ttin g y o u r hands d ir t y and y o u ju s t need t o how w ill fin d im p o rta n t, ju s t t o make sure I h a d n 't fo rg o tte n
t o use y o u r hands. O th e r tim e s , y o u m ig h t be research­ a n y th in g (a n d I'Ll fre e ly a d m it t h a t I'd fo rg o tte n th e spe­
in g o r Learning a to p ic w here y o u h a v e n 't g o tte n y o u r c ific s o f F ill F a cto r). I also kn o w som eone w ho's c o n s id e r­
hands d irty . ALL Learning is k in e tic in som e w a y— a person in g a career in th is in d u s try . I'v e been su cce ssfu l in m y
c o u ld read a b o o k on ho w t o p e rfo rm open h e a rt surgery career: I'v e made m any m ista ke s, and I'v e Learned hard
100 tim e s and be a b le to q u o te each Line in th e b o o k, lessons as w e ll! I w a n te d to lo o k back on w h a t areas o f
fo r in s ta n c e . W ell, I'm n o t sa y in g t h a t im p le m e n tin g a kn o w le d g e have helped me to be succe ssful.
Type 2 c h a n g in g d im e n s io n is open h e a rt surgery, b u t
th e m ore y o u can d e m o n s tra te to o th e rs t h a t y o u CAN I'v e been ta lk in g to a d e v e lo p e r t h a t I m e n to re d fo r a fe w
do s o m e th in g .... As som eone w ho 's in te rv ie w e d p e o p le , I years. T h e ir f ir s t response was, "W ow , y o u 're re a lly tr y in g
m ig h t fin d som eone's p e rs o n a l exa m ple (a go od e xa m ple ) to expose in te r v ie w e r s !!!" As Eric Id le said t o J o h n Cleese
o f im p le m e n tin g a Type 2 SCD, o r som eone b e in g a b le to d u rin g th e fam ous "N ud ge Nudge W in k W in k " s k it: "O h no,
open tw o qu e ry w in d o w s w ith a te s t ta b le t o d e m o n s tra te no , no , (p a u s e ), YES!"
READ COMMMITTED SNAPSHOT, to be ve ry c o m p e llin g .
There are to p ic s I covered in th is a rtic le in m ore d e ta il
th a n o th e rs . I w e n t in t o a f a ir a m o u n t o f d e ta il on th e
One Thing I Won't Talk About S n a p s h o t Is o la tio n Level, b u t o n ly b r ie fly ta lk e d a b o u t
(But One Final Thing That I Will) Change Data C apture and lo g g in g . There are o th e r w eb
There are o th e r g re a t to o ls and te c h n o lo g ie s th a t database a rtic le s o u t th e re , in c lu d in g som e fro m me. As I'v e lin k e d
developers use. One t h a t comes to m ind is P ython. Database in th is a rtic le , th e re w ere som e to p ic s t h a t I'v e p re v io u s ly
developers w ho also w ork on th e .NET side w ill som etim es covered in CODE M agazine and d id n 't w a n t to re p e a t.
use E n tity Framework. Those w ho w ork on th e ETL side m ig h t
use th ird -p a rty to o ls such as COZYROC and possibly d iffe re n t Kevin S. G o ff
M aster Data M anagem ent to o ls . The List goes on and on. CODE

58 Stages o f Data:The DNA o f a Database Developer, Part 1 codemag.com

 ONLINE QUICK ID 2405071

From SOAP to REST to GraphQL

Sim pie Object Access Protocol (SOAP) can be used to build web services that support interoperability between different platforms
and technologies. REST (an acronym for Representational State Transfer) is another popular way of building lightweight APIs
that can run over HTTP. As an open-source query language, GraphQL promises a more potent way of accessing information

th a n SOAP o r REST a lo n e . This a rtic le aim s to p ro v id e a t h a t a llo w s d iffe r e n t a p p lic a tio n s to in te r a c t w ith one a n ­
co m p re h e n sive o v e rv ie w o f th e e v o lu tio n o f w eb A PIs, o th e r o ve r a n e tw o rk by Leveraging XML as th e message
e x p lo rin g th e tra n s itio n fro m SOAP to REST, and fin a lly fo rm a t. You can ta k e a d va n ta g e o f SOAP t o b u ild in te r o p ­
to GraphQL. I t w ill d e lve in t o th e m o tiv a tio n b e h in d each era b le w eb services t h a t w o rk w ith d is p a ra te te c h n o lo g ie s
a r c h ite c tu ra l s ty le , and t h e ir c h a ra c te ris tic s , b e n e fits , and p la tfo rm s . The s tru c tu re and c o n te n t o f XML m es­
and draw backs. By u n d e rs ta n d in g th e pro g re ssio n fro m sages, as w e ll as a s e t o f c o m m u n ic a tio n g u id e lin e s , are
SOAP to REST and th e em ergence o f GraphQL, d e velop ers o u tlin e d in a SOAP d o c u m e n t.
can m ake in fo rm e d d e cisio n s w hen c h o o s in g th e r ig h t A P I
design fo r th e ir p ro je c ts . N ote t h a t ASP.NET Core d o e s n 't have any b u ilt - in s u p p o rt
fo r SOAP. R ather, th e .NET Fram ew ork p ro vid e s b u ilt - in Joydip Kanj i lai
I f y o u 're t o w o rk w ith th e code exam ples discussed in th is s u p p o rt fo r w o rk in g w ith ASMX and WCF. U sing th ir d - p a r ­ j oy di p ka n ji LaL@y a h o o . co m
a rtic le , y o u need th e fo llo w in g in s ta lle d in y o u r syste m : t y lib ra rie s , y o u can s t i l l b u ild a p p lic a tio n s t h a t Leverage
Jo yd ip K a n jila l is an MVP
SOAP in ASP.NET Core. Figure 1 d e m o n stra te s how SOAP
• V isu a l S tu d io 2022 w orks. (2 0 0 7 -2 0 1 2 ), softw are
a rc h ite c t, author, and
• .NET 8 .0
speaker w ith more tha n
• ASP.NET 8 .0 R un tim e
Anatomy of a SOAP Message 20 years o f experience.
He has more th a n 16 years
I f y o u d o n 't a lre a d y have V is u a l S tu d io 2022 in s ta lle d on A SOAP (S im ple O b je ct Access P ro to co l) message is an XML-
o f experience in M icrosoft
y o u r c o m p u te r, y o u can d o w n lo a d i t fro m here: h t t p s : / / based s tru c tu re used to exchange in fo rm a tio n betw een web
.NET and its related
v is u a ls tu d io .m ic ro s o ft.c o m /d o w n lo a d s /. services across diverse n e tw orks and p la tfo rm s . A ty p ic a l
tech no lo gies. Jo yd ip has
SOAP message is com prised o f several ele m en ts t h a t d e fin e
authored e ig h t books,
In th is a rtic le , I ’l l exa m ine th e fo llo w in g p o in ts : th e message's stru c tu re , c o n te n t, and o p tio n a l fe a tu re s. more th a n 500 articles,
SOAP messages are designed to be e xte n sib le , n e u tra l, and and has reviewed more
• SOAP, REST, and GraphQL and t h e ir b e n e fits in d e p e n d e n t o f any s p e c ific p ro g ra m m in g m o del o r tra n s ­ th a n a dozen books.
• The key differences betw een SOAP, REST, and GraphQL p o rt p ro to c o l, ty p ic a lly HTTP o r HTTPS.
• The b e n e fits and drawbacks o f SOAP, REST, and GraphQL
• H ow to use each o f th e s e to o ls in e n te rp ris e apps A ty p ic a l SOAP message com prises fo u r key e le m e n ts, as
S ^E ■ ■■ ■ ■ ■ ■_

show n in Figure 2.
A fte r g a in in g th is kn o w le d g e , y o u 'll b u ild th re e a p p lic a ­
tio n s : one each u sin g SOAP, REST, and GraphQL. • E nvelope
• H eader (o p tio n a l)
• Body
What Is Simple Object Access • F a u lt
Protocol (SOAP)?
S im p le O b je c t Access P ro to c o l (SOAP) is a c o m m u n ic a tio n T his n e x t s h ip p e t is ho w th e s tru c tu re o f a ty p ic a l SOAP
p ro to c o l fo r d a ta exchange in a d is tr ib u te d e n v iro n m e n t message lo o ks:

SOAP Message

SOAP Response

Application Server Client Applications

(SOAP Sender) (SOAP Receiver)

Figure 1: S im p le O b je c t Access P ro to c o l (SOAP) a t w o rk

codemag.com From SOAP to REST to GraphQL

 <?xml version="1.0"?> Example SOAP Request and Response
The form at o f a ty p ic a l SOAP request Looks like th is :
<soap:Envelope
xmlns:soap= POST /<host>:<port>/<context>/
"http://www.w3.org/2003/05/soap-envelope/" database ID> HTTP/1.0
soap:encodingStyle= Content-Type: text/xml; charset=utf-8
"http://www.w3.org/2003/05/soap-encoding">
<?xml version="1.0"?>
<soap:Header> <env:Envelope xmlns:env=
<!-- SOAP Header (Optional) --> "http://schemas.xmlsoap.org/soap/envelope/">
</soap:Header>
<env:Header>
<soap:Body> </env:Header>
<!-- SOAP Body -->
<soap:Fault> <env:Body>
<!-- SOAP Fault --> </env:Body>
</soap:Fault>
</soap:Body> </env:Envelope>

</soap:Envelope> The fo llo w in g code snippet illustra te s how a ty p ic a l SOAP

request looks:

SOAP Envelope
Every SOAP message is encapsulated inside a roo t elem ent POST /ProductPrice HTTP/1.1
called SOAP Envelope. I t defines the XML namespace and
Host: www.example.org
contains tw o mandatory child elements: the SOAP Header
Content-Type: application/soap+xml;
and the SOAP Body.
charset=utf-8
Content-Length: nnn
<soap:Envelope
<?xml version="1.0"?>
xmlns:soap=
"http://www.w3.org/2003/05/soap-envelope/" <soap:Envelope
xmlns:soap=
soap:encodingStyle=
"http://www.w3.org/2003/05/soap-encoding"> "http://www.w3.org/2003/05/soap-envelope/"
soap:encodingStyle=
"http://www.w3.org/2003/05/soap-encoding">
SOAP Header
The SOAP Header is an o p tio n a l elem ent th a t contains
<soap:Body xmlns:m=
a d d itio n a l in fo rm atio n or metadata about the SOAP mes­
"http://www.abcxyz.org/product">
sage, such as au the ntica tion credentials, security tokens,
<m:GetProductPrice>
or rou ting instructions.
<m:ProductCode>HP_Envy_i9</m:ProductCode>
</m:GetProductPrice>
<soap:Header>
</soap:Body>
<!-- Optional SOAP Header elements -->
</soap:Header>
</soap:Envelope>

SOAP Body The form at o f a ty p ic a l SOAP response looks Like th is :

The SOAP Body represents the main body o f the SOAP
message, which contains the data or parameters fo r the HTTP/1.0 200 OK
method being sent to the web service. I t should be noted Content-Type: text/xml; charset=utf-8
th a t the SOAP Body elem ent is mandatory. I t can have
one or more child elements th a t represent the actual pay­ <?xml version="1.0"?>
load and one or more fa u lt elements in the event o f an <env:Envelope xmlns:env=
error. "http://schemas.xmlsoap.org/soap/envelope/">

<soap:Body> <env:Header>
<!-- SOAP Body content --> </env:Header>
</soap:Body>
<env:Body>
SOAP Fault </env:Body>
During the processing o f a SOAP message, an o p tio n a l
elem ent called SOAP Fault can be used to convey error or </env:Envelope>
fa u lt in fo rm atio n back to the c lie n t in case o f errors or
exceptions th a t occur during the process. And here's how a SOAP response to the above SOAP re­
quest Looks:
<soap:Fault>
<!-- Fault details --> HTTP/1.1 200 OK
</soap:Fault> Content-Type: application/soap+xml;

60 From SOAP to REST to GraphQL codemag.com

charset=utf-8
Content-Length: nnn

<?xml version="1.0"?>

<soap:Envelope
xmlns:soap=
"http://www.w3.org/2003/05/soap-envelope/"
soap:encodingStyle=
"http://www.w3.org/2003/05/soap-encoding">

<soap:Body Figure 3: Interactions between components in a web service

xmlns:m="http://www.abcxyz.org/product">
<m:GetProductPriceResponse>
<m:Price>6500.00</m:Price> • Service requester or th e service consumer: This
</m:GetProductPriceResponse> com ponent is responsible fo r requesting a resource
</soap:Body> from the service provider and, sim ila r to the service
</soap:Envelope> provider, i t encompasses the application, the p la t­
form , and the middleware.
• Service registry: This is an o p tio n a l com ponent
WSDL, UDDI, and Binding th a t resides in a central Location so th a t service
This section provides a b rie f overview o f the WSDL and providers can publish service descriptions and the
UDDI. I t also b rie fly discusses SOAP binding. service requesters or service consumers can Look up
those service descriptions.
Web Services Description Language (WSDL)
Web Services Description Language, or WSDL, is a lan­
guage based on XML used fo r the purpose o f describing Strategies for Building a SOAP Service
and locating web services. A standard form at is used to To build a SOAP service, you can use any o f the fo llo w in g :
describe an interface to a web service th a t includes op­ WCF connected service, th ird -p a rty Libraries, or manual
erations, in pu ts, o u tp u t parameters, and the message im plem entation. Lets have a look.
form at.
WCF Connected Service
Universal Description, Discovery, and Integration (UDDI) A Connected Service in Windows Communication Founda­
Universal Description, Discover, and In te g ra tio n , or UDDI, tio n (WCF) represents a feature in Visual Studio th a t fa­
represents a platform -independent registry intended to c ilita te s the use o f and access to web services, including
provide a standard mechanism fo r service discovery, regis­ SOAP-based services. Using it , you can create c lie n t code
tra tio n , and versioning to enable organizations to publish capable o f com m unicating w ith a web service seamlessly.
and discover web services on the in te rn e t.
Third-Party Libraries
A UDDI registry is independent o f any platform intended You can also create SOAP services using th ird -p a rty lib ra r­
to provide a standard mechanism fo r the discovery, reg­ ies such as Apache CXF, JAX-WS (Java API fo r XML Web
is tra tio n , and versioning o f services, thereby enabling Services), and Spring-WS (Spring Web Services). You can
businesses to publish and discover web services on the also leverage Windows Communication Foundation (WCF)
in te rn e t. The key components o f the SOAP architecture fo r bu ildin g SOAP services in .NET environm ent.
work to g ethe r to define a standardized message form at
fo r exchanging structured data between applications us­ Manual Implementation
ing d iffe re n t tran spo rt protocols. SOAP messages can be manually constructed and pro­
cessed using programming Languages th a t support XML
SOAP Binding parsing and SOAP protocols, such as C# or Java. To do
SOAP binding defines how SOAP messages are tran sm itte d th is , fo llo w these steps:
using tran spo rt protocols, such as HTTP, SMTP, or TCP. This
docum ent specifies the message form at, SOAP encoding 1. Define the service interface, operations, messages,
style, and binding rules th a t must be follow ed when send­ and bindings in the WSDL document.
ing messages over SOAP. 2. Leverage the programming Language o f your choice
to im plem ent the service logic.
3. Use XML parsing Libraries or manually handling XML
SOAP Web Services Architectural serialization and deserialization fo r SOAP messages
Components is the o p tio n .
The SOAP web services architecture thrives on the syn­ 4. Keep an eye on the HTTP server fo r SOAP requests
ergy among the fo llo w in g three components, as shown and responding appropriately.
in Figure 3:

• Service provider: This is the com ponent th a t pro­ DataContract and ServiceContract
vides the web service and encompasses the applica­ In SOAP, DataContract and ServiceContract are key con­
tio n its e lf, the platform on which the application cepts used to define the structure o f data and the opera­
executes, and the middleware. tions supported by the service.

codemag.com From SOAP to REST to GraphQL 61

 D ataC o ntra ct 1. S tart the Visual Studio 2022 IDE.
In a DataContract, the terms and conditions used in the 2. In the "Create a new p ro je ct" window, select "ASP.
data exchange between data providers and consumers are NET Core Web A P I" and click Next to move on.
ou tlin ed along w ith the form at, structure, quality, seman­ 3. Specify the project name as S0AP_Demo and the
tics, etc. It's ty p ic a lly defined using XML Schema D efini­ path where i t should be created in the "Configure
tio n (XSD) or a sim ilar language in SOAP. These could your new p ro je ct" window.
be complex types th a t are passed between service opera­ 4. I f you w ant the solution file and project to be cre­
tions. You should decorate your classes or structs w ith the ated in the same directory, you can o p tio n a lly check
DataContract a ttrib u te to indicate th a t they are serializ­ the "Place solution and project in the same direc­
able and understandable by the service provider and the to ry " checkbox. Click Next to move on.
service consumer. Leverage the DataMember a ttrib u te to 5. In the next screen, specify the ta rg e t framework and
decorate the properties or methods o f classes or structs authe ntica tion type as w ell. Ensure th a t the "Config­
th a t should take part in the serialization process. ure fo r HTTPS," "Enable Docker Support," "Do n o t use
to p -le v e l statem ents," and the "Enable OpenAPI sup­
Here's an example o f a ty p ic a l data contract: p o rt" checkboxes are unchecked because you w o n't
use any o f these in th is example.
[DataContract] 6. Remember to Leave the Use controllers checkbox checked
public class MyClass because you won't use minimal API in this example.
{ 7. Click Create to com plete the process.
[DataMember]
public int Id { get; set; } A new ASP.NET Core Web API project is created. You'Ll use
th is project to build the SOAP service in ASP.NET Core.
[DataMember]
public string Text { get; set; } In sta ll NuG et Package(s)
} So fa r so good. The next step is to in s ta ll the necessary
NuGet Package(s). To in s ta ll the required package in to
ServiceContract your project, rig h t-c lic k on the solution and then select
SOAP web services expose operations and methods Manage NuGet Packages fo r S o lu tio n .... Now search fo r
thro ug h th e ServiceContract th a t specifies the set o f the package named SoapCore in the search box and in ­
available operations, th e ir in p u t parameters, and return s ta ll it . A lternatively, you can type the commands shown
types, i.e ., the type o f the return values. A ServiceCon­ below a t the NuGet Package Manager Command Prompt:
tra c t consists o f an in te rfa ce or class im plem ented in
the server-side code and acts as a b lu e p rin t fo r service PM> Install-Package SoapCore
im p le m e n ta tio n , specifying the operations clien ts can
invoke. You can also in s ta ll th is package by executing the fo llo w ­
ing commands a t the Windows Shell:
An O perationcontract represents an in d iv id u a l opera­
tio n w ith in a ServiceContract and specifies the operation dotnet add package SoapCore
name, in p u t parameters, and o u tp u t types. You must ap­
ply the O perationcontract a ttrib u te to a service operation Create the D a ta C ontract
to in dicate th a t clients can access the operation. To create the data contract, create a new class named
Customer in a file called Customer.es and w rite the fo l­
The fo llo w in g code snippet illustra te s a service contract: low ing code in there:

[ServiceContract] [DataContract]
public interface IMyDemoService public class Customer
{ {
[Operationcontract] [DataMember]
string GetText(int id); public int Id { get; set; }
} [DataMember]
public string FirstName { get; set; }
[DataMember]
Im p le m e n t a SOAP Service in public string LastName { get; set; }
ASP.NET Core [DataMember]
In th is section, I 'l l examine how to bu ild a SOAP service public string Address { get; set; }
in ASP.NET Core. The section th a t follow s outlines the }
series o f steps needed to create a new ASP.NET Core Web
API project in Visual Studio. Create the Custom erRepository
The CustomerRepository class extends the ICustomerRe-
Create a NewASP.NET Core 8 Project in Visual Studio 2022 pository interface and im plem ents its methods, as shown
You can create a project in Visual Studio 2022 in several in the code given in Listing 1.
ways. When you Launch Visual Studio 2022, you’ll see the
S tart window. You can choose "Continue w ith o u t code" Create the Service C ontract
to Launch the main screen o f the Visual Studio 2022 IDE. To create a service contract, create an interface called
ICustomerService and w rite the code given in Listing
To create a new ASP.NET Core 8 Project in Visual Studio 2022: 2 in there. The CustomerService class extends the ICus-

62 From SOAP to REST to GraphQL codemag.com

tomerService interface and im plem ents its methods. Once the Program.es file is given in Listing 3 fo r your refer­
you've created the ICustomerService interface, create a ence.
new class named CustomerService and w rite the code
given in Listing 2 in there. Create the SOAP C lient
You'll now create a sim ple Console application to consume
Configure the SOAP Service the SOAP service you created earlier. The SOAP c lie n t ap­
Lastly, you should configure your SOAP service to run it. plica tion consumes the SOAP service and displays the
To do th is , add ICustomerRepository and ICustomerSer­ data retrieved a t the Console window.
vice instances to the container using the fo llo w in g code
snippet: You can create a project in Visual Studio 2022 in several
ways. When you launch Visual Studio 2022, y o u 'll see the
buiIder.Servi ces.AddScoped S tart window. You can choose "Continue w ith o u t code"
dCustomerService, to Launch the main screen o f the Visual Studio 2022 IDE.
CustomerService>();
buiIder.Services.AddScoped To create a new Console A pplication Project in Visual Stu­
clCustomerRepository, dio 2022:
CustomerRepository>();
1. S ta rt the Visual Studio 2022 IDE.
This enables you to use dependency in je c tio n to retrieve 2. In th e Create a new p ro je c t window, select Console
these instances a t runtim e. The com plete source code o f App, and click Next to move on.

Listing 1: The ICustomerRepository interface

public interface ICustomerRepository {

public Task < List < Customer >> GetCustomersO; }.
public Task < Customer > GetCustomer(int id); new CustomerO {
} Customer_Id = 3, FirstName = "Carlton",
LastName = "Kramer", Address = "New York, USA"
public class CustomerRepository: ICustomerRepository { }
private readonly List < Customer > customers = };
new List < Customer > () {
new Customer() { public async Task < List < Customer >> GetCustomersO {
Customer_Id = 1, FirstName = "Rob", return await Task.FromResult(customers);
LastName = "Miles", Address = "Boston, USA" }
public async Task < Customer > GetCustomer( int id) {
}. return await Task.FromResult
new Customer() { (customers.FirstOrDefault(x => x.CustomerJd == id));
Customer_Id = 2, FirstName = "Lewis", }
LastName = "Walker", Address = "London, UK" }

Listing 2: The ICustomerService interface

[ServiceContract] (ICustomerRepository CustomerReposi tory)

public interface ICustomerService {
{ _customerRepository = CustomerRepository;
[Operationcontract] }
Task<List<Customer>> GetCustomersO; public async Task<List<Customer» GetCustomersO
{
[Operationcontract] return await -CustomerRepository.GetCustomersO;
Task<Customer> GetCustomer(int id); }
} public async Task<Customer> GetCustomer(int id)
{
public class CustomerService : ICustomerService return await -CustomerRepository.GetCustomer(id);
{ }
private readonly ICustomerRepository _customerRepository;
public CustomerService

Listing 3: Configuring SOAP endpoints in the Program.es file

using SOAP—Demo; // Configure the HTTP request pipeline.

using SoapCore; app.UseRouti ng();
app.UseAuthori zation();
var builder = WebApplication.CreateBuilder(args); app.MapControllers();
// Add services to the container. app.UseEndpoints(endpoints =>
buiIder.Services.AddScoped {
dCustomerService, CustomerService>(); _ = endpoints.UseSoapEndpoint
buiIder.Services.AddScoped dCustomerServi ce>("/CustomerService.asmx",
<ICustomerRepository, CustomerRepository>(); new SoapEncoderOptions(),
buiIder.Services.AddControllers(); SoapSerializer.XmlSerializer);
});
var app = builder.BuildO;
app.Run();

codemag.com From SOAP to REST to GraphQL 63

 X You'll observe th a t a file named References.cs has been
added to the c lie n t ap plication. You can now w rite the
Add new WCF Web Service service reference
fo llo w in g piece o f code to invoke the GetCustomers ser­
Specify the service to add
vice operation asynchronously.
To m * * lift of t v i U b e o r a tptcrfk t t i w . f < w a w m m UR1 and taltct Go To And a/a<Ubi« sot vk m >n tha solution select
Lhucw To load a »««vke riw ta d a u fru*t> « W SDt Me. MrWit 6> v*> e
ICustomerService soapServiceChannel =
new CustomerServiceClient
(CustomerServi ceClient.Endpointconfiguration.
Basi cHttpBi ndi ngJCustomerServi ce_soap);
var response = await
IC u sto Te'S civK o
soapServi ceChannel.GetCustomersAsyncO;

The fo llo w in g code snippet shows the com plete source

code o f the Program.es file o f the c lie n t application.

using CustomerServiceReference;

ICustomerService soapServiceChannel =
new CustomerServiceClient
(CustomerServi ceCli ent.
Back Next Cancel
Endpointconfiguration.
Basi cHttpBi ndi ngJCustomerServi ce_soap);
Figure 4: Adding a new Service Reference var response = await soapServiceChannel.GetCustomersAsyncO;

foreach (Customer customer in response)

Console.WriteLine(customer.FirstName);
Service reference configuration progress
When you run the ap plication, the fir s t names o f the cus­
Importing w eb s rm c e metadata
ScaffoUmg s e m e * re 'e ’ence c o d e .. tomers w ill be displayed a t the console window.
N jm o e r c l service endpoints found 1
Updating project
Dona Call the SOAP Service from Postm an
O Succeufu ly added w n x t referenced)
You can also call the GetCustomers service operation us­
ing Postman, as shown in Figure 6.

Automatic ally dose when succeeded From SOAP to REST

Representational State Transfer (REST) is a Lightweight,
Ckxe
and scalable approach to build web services th a t can run
over HTTP. It's often used fo r bu ild in g d istrib u te d sys­
Figure 5: SOAP service reference added tems and APIs due to its sim p licity, fle x ib ility , and wide
adoption among developers. REST is a set o f architectural
constraints.
3. Specify the p ro je ct name as SOAP_Cient and the
path where i t should be created in the Configure RESTful systems adhere to a set o f constraints th a t stan­
you r new p ro je ct window. dardize the com m unication between components, en­
4. I f you w ant th e s o lu tio n file and p ro je ct to be cre­ hancing sca la b ility and performance. By leveraging HTTP
ated in the same directory, you can o p tio n a lly check methods such as GET, PUT, POST, and DELETE, REST en­
the Place s o lu tio n and p ro je c t in the same d irectory ables the creation o f w ell-defined and predictable APIs.
checkbox. Click Next to move on. This approach allows Loose coupling between c lie n t and
5. In th e next screen, specify the ta rg e t fram ework server, prom oting fle x ib ility and ease o f maintenance in
you'd Like to use fo r you r console a p p lica tio n . d istrib u te d systems.
6. Click Create to com plete th e process.
This n o t only promotes re lia b ility and resilience, b u t also
Once you r Console a p p lica tio n is ready, you can fo llo w allows sca la b ility and e ffic ie n t com m unication between
these steps to add a reference to you r SOAP service in to components. Figure 7 shows a ty p ic a l REST-based a p p li­
th e c lie n t a p p lica tio n . cation a t work. By understanding the principles o f REST,
developers can design robust and flexible systems th a t
1. Right click on the clie n t application and select Add are scalable and high perform ant.
-> Service Reference, as shown in Figure 4.
2. Click on Next three tim es and le t the default selec­ Resources
tions be used fo r th is example. A fundam ental te n e t o f REST is resources, which are
3. Click on Subm it to in itia te the add reference process. distinguished by unique URLs and may be managed via
conventional HTTP methods such as GET, POST, PUT, and
Once the reference to the SOAP service has been added DELETE. This approach allows clients to access and m odify
successfully, the fo llo w in g screen w ill be displayed, as representations o f these resources through w ell-defined
shown in Figure 5. interfaces. Resources can be represented in various for-

64 From SOAP to REST to GraphQL codemag.com

Figure 6: The SOAP response o f the GetCustomers service operation in Postman

mats, allow ing more fle x ib ility in how data is m anipu­

lated and transferred.

Common Misconceptions about REST GET

In th is section, I 'l l examine a few common misconcep­ PUT
tion s related to REST among the developer com m unity.
POST

REST Is a P rotocol DELETE

I t should be noted th a t REST is n o t a standard or a pro­ PATCH
to c o l. Representational State Transfer (REST) refers to an
a rch itectu ra l style and a set o f architectural constraints
used fo r developing networked applications th a t defines a
set o f guidelines and principles fo r developing web servic­
es th a t are scalable, m aintainable, and loosely coupled.
Figure 7: REST application a t work
REST Is O nly Used fo r Web Services
A lthough REST was o rig in a lly designed fo r creating web
services, i t can also be used fo r other types o f applica­ the a rch itectu ra l style, in clu din g statelessness, caching,
tion s such as m obile apps or loT devices. As Long as the uniform interface, etc.
principles o f statelessness, client-server architecture, and
resource-based com m unication are follow ed, any type o f URLs M u st C ontain Nouns O nly
application can be b u ilt using REST. A nother m isconception about REST is th a t i t can only be
used w ith HTTP. A lthough RESTful APIs ty p ic a lly use HTTP
REST Requires the Use o f HTTP as the underlying protocol, REST its e lf is n o t tie d to any
A lthough HTTP is com m only used in conjunction w ith specific protocol and can be im plem ented over other pro­
REST due to its widespread adoption and support fo r tocols like CoAP or WebSocket.
various request methods, it's n o t a requirem ent. The p rin ­
ciples o f resource id e n tific a tio n and m anipulation are ap­
plicable to any network protocol. Key Principles of REST
There are several key principles th a t underpin REST archi­
Every API th a t Uses HTTP Is A u to m a tic a lly Considered tecture, which are Listed in th is section. By adhering to
RESTful these key principles, developers can design scalable, re li­
No, n o t actually. Just because an API uses HTTP doesn't able, and e ffic ie n t web services th a t meet the demands o f
mean i t follow s the principles o f REST. A genuinely REST- today's applications as far as performance and fle x ib ility
fu l API should adhere to a ll the constraints set o u t by is concerned.

codemag.com From SOAP to REST to GraphQL 65

 Resource-Based In a ty p ic a l client-server architecture, the c lie n t and the
In REST, a resource is an object, data, or service th a t a server are decoupled and they do n't have any knowledge
c lie n t may access across a network, usually the in te rn e t, o f each other, thereby enabling them to grow and evolve
using a specified URL. Resource id e n tifie rs in a RESTful independent o f each other.
system are known as URIs (Uniform Resource Id e n tifie rs ),
and they are conceptual e n titie s or data representations. C acheable
In a RESTful architecture, you can id e n tify resources us­ Resources should be cacheable to improve network e fficien­
ing URIs. Resources are concepts th a t can be represented cy. Each response should state whether i t is cacheable on
in any form at such as HTML, XML, JSON, etc. the clien t side and for how long. When the clien t requests
data in the future, i t retrieves the data from its cache,
Code on D e m a n d thereby elim inating the need to transm it the request to the
A RESTful architecture provides support fo r an o p tio n a l server again fo r future requests. When managed effectively,
feature th a t allows code to be downloaded and executed caching reduces clien t and server tra ffic, enhancing avail­
as applets or scripts to extend c lie n t fu n c tio n a lity . The a b ility and performance. However, you should take proper
number o f features th a t need to be pre-im plem ented is measures to ensure th a t clients don't have stale data. The
reduced, sim p lifyin g the c lie n t experience. The a b ility to server can include caching-related headers (e.g., Cache-
download features a fte r deploym ent improves the exten­ Control or Last-Modified) in the response to indicate to the
s ib ility o f the system. By delivering executable code to clien t how long the response can be cached.
the clie n t, the server may enhance the client's ca p a b ility
via the code on demand feature. When you f i l l o u t any
registration form on a site, your web browser alerts you Benefits o f Using REST
i f there are errors. For example, when you f i l l in a regis­ The fo llo w in g are the benefits o f REST a t a glance:
tra tio n form , your browser displays any errors w hile you
type, such as incorrect SSN numbers, em ail addresses, etc. • Scalability: RESTful architectures are in he ren tly
scalable due to th e ir stateless nature, allow ing fo r
Stateless easy scaling o f services to meet changing demands
REST is stateless, requiring each c lie n t request to provide and making i t easier to handle a large number o f
a ll in fo rm atio n needed fo r processing as part o f query requests.
parameters, request headers or URL I t should be noted • Sim plicity: W ith its emphasis on standard HTTP
th a t in a ty p ic a l RESTful architecture, the server doesn't methods and status codes, REST sim plifies the com­
retain any clien t-sp ecific in fo rm atio n between requests, m unication process between clients and servers.
thus enabling im proved sca la b ility and load balancing by • Flexibility: The a b ility to work w ith d iffe re n t data
allow ing any server to process any c lie n t request. representations and to support various c lie n t types
adds fle x ib ility to RESTful services.
Layered A rchitecture • Performance: By caching resources, you can reduce
Requests and responses to REST APIs are routed through client-server interactions, which can greatly im ­
several Layers spread across m u ltip le tiers. A Layered sys­ prove the performance o f your application.
tem architecture is one in which the application is s p lit • In tero p erab ility: REST APIs work over standard pro­
across various Layers to isolate presentation, application tocols Like HTTP, enabling seamless com m unication
processing, and data management in a Layered architec­ between d iffe re n t systems regardless o f th e ir im p le­
ture. These Layers work to g e th e r to f u lf ill clie n t requests m entation details.
w hile clients are unaware o f them . I t helps i f you design • Technology Agnostic: REST APIs are technology ag­
your RESTful architecture to s p lit your RESTful services nostic, enabling you to w rite both server and c lie n t
across m u ltip le layers, thereby fostering the separation o f applications in d iffe re n t programming languages.
concerns in your application. The underlying technology can also be changed on
e ith e r side o f the com m unication, i f needed, w ith ­
U niform Interface o u t a ffe ctin g the fu n ctio n a lity .
RESTful architecture encourages uniform and standard­
ized interfaces fo r in te ra ctio n w ith a resource as part o f
its basic principles. The interface ty p ic a lly consists o f H ow Does REST Work?
fou r main HTTP methods: GET, POST, PUT, and DELETE. In itia lly , the clie n t sends a request to the server over
RESTful services should have a uniform interface. In a HTTP in order to in itia te com m unication w ith the server.
RESTful architecture, the server transm its in fo rm atio n to The server acknowledges and processes the request and
a c lie n t in a standard form at. In RESTful architecture, a then produces a response, as appropriate. Responses are
form atted resource is known as a representation. Note usually in the form o f data, such as JSON or XML, which
th a t the form at o f a resource may d iffe r from its in te rn a l the c lie n t can display or use. REST APIs use predefined
representation on the server. A server can, fo r instance, methods like GET, PUT, POST, DELETE to perform diverse
store data in te x t and send i t in HTML form at. In a REST- operations on the server. These methods correspond to
fu l architecture, by decoupling im plem entations from the various actions, such as retrieving data, updating old
services they provide, they can evolve independently. data, creating new data, and deleting old data.

Client-Server A rchitecture
As a rule o f thum b, a RESTful architecture should be
W hat Are REST APIs?
based on a client-server architecture. A lthough the c li­ H ow Do They Work?
en t requests resources from the server, the server pro­ REST APIs communicate data between c lie n t and server
vides resources as appropriate to the authorized clients. using HTTP requests. Once a c lie n t sends a request, the

66 From SOAP to REST to GraphQL codemag.com

server acknowledges and processes i t and then sends an interconnected devices. As the web in du stry continues to
appropriate response to the clien t. Responses are usually evolve, REST's fle x ib ility and c o m p a tib ility w ith d iffe re n t
in the form o f data, such as JSON or XML, which the c lie n t programming Languages make i t w e ll-positioned to meet
can display or use. REST APIs use predefined methods like the demands o f modern web developm ent practices.
GET, POST, PUT, and DELETE to perform diverse operations
on the server. The methods apply to various actions, such
as retrieving data, creating new data, updating old data,
Im p le m e n tin g a RESTful A pplication
and deleting old data. Using these principles and m eth­ in ASP.NET Core
ods, REST APIs can e ffe ctive ly com m unicate and transfer Let's now build a RESTful web application in ASP.NET Core.
data between applications. A REST API encompasses a Follow the steps ou tlin ed in a previous section where
collection o f guidelines and standards th a t can help you you've b u ilt a ASP.NET Core Web API application in Visual
build applications th a t can in te ra c t and share in fo rm a­ Studio. The only difference is th a t y o u 'll build a m inim al
tio n over the in te rn e t. I t adheres to REST principles, a API application in Visual Studio by Leaving the Use con­
stateless a rch itectu ra l approach, to develop scalable and trollers checkbox unchecked. You'Ll use th is project to
adaptable web services. bu ild the RESTful application in ASP.NET Core.

Create the M o d e l Class

Challenges o f REST Assuming th a t the m inim al API application has been cre­
A lthough REST offers many advantages, some challenges ated, create a new class named Product in a file having
exist, such as m aintaining statelessness leading to in ­ the same name w ith a .cs extension and w rite the fo llo w ­
creased network overhead and designing consistent and ing code in there:
m eaningful URI structures can be complex in large-scale
applications. public class Product
{
Here are the key challenges o f RESTful architecture: public int Product_Id { get; set; }
public string Product_Name { get; set; }
• Lim ited support fo r perform ing complex opera­ public string Description { get; set; }
tions: It's im p o rta n t to note th a t the REST API can public string SKU { get; set; }
only be used fo r CRUD (Create, Read, Update, De­ public decimal Price { get; set; }
lete) operations through a lim ite d set o f HTTP m eth­ }
ods (GET, PUT, POST, DELETE). Performing complex
operations may require m ultiple requests or custom For the sake o f s im p lic ity and brevity, I 'l l skip oth er model
endpoints. classes here.
• No standard error handling mechanism: There's no
standardized error handling in REST. There can be Create the ProductRepository
a range o f REST im plem entations fo r im plem enting The ProductRepository abstracts a ll calls to the database.
consistent error handling mechanisms, such as error The ProductRepository class extends the IProductReposi-
codes, error messages, and error form ats. to ry interface and im plem ents its methods as shown in
• Over-fetching and under-fetching: REST APIs Listing 4.
ty p ic a lly return a fixed representation o f resources.
When clients only require a subset o f the resource Create the RESTful API
data or need a d d itio n a l data n o t included in the Finally, let's create the API endpoints in the Program.es
response, i t can result in inefficiencies. file . Remember, th is is a M inim al API application, so you
• Versioning: As REST APIs evolve, in troducing chang­ don't use any APIController class here. The follow ing code
es can break existing c lie n t im plem entations. When snippet shows how you can create the API endpoints:
many clients consume the API a t the same tim e,
m aintaining backward c o m p a tib ility and versioning app.MapGet("/getproducts", async
can be challenging. (IProductRepository productRepository)
• Security issues: Security challenges exist when using => await productRepository.GetProducts());
REST APIs, such as data exposure, unauthorized ac­
cess, and protection against XSS and CSRF attacks. Im ­ app.MapGet("/getproduct/{id:int}", async
plementing proper security measures, such as authen­ (IProductRepository productRepository, int id)
ticatio n, authorization, and encryption, is imperative. => await productRepository.GetProduct(id)
is Product product ?
Results.Ok(product) : Results.NotFoundO);
The Future o f REST
W ith technology advancing a t such a lig h tn in g -fa s t pace, app.MapPost("/addproduct" , async
REST seems to have a b rig h t future. W ith its s im p lic ity and (IProductRepository productRepository,
scalability, REST is expected to remain a fundam ental ar­ Product product) =>
ch ite ctu ra l style fo r designing networked applications. As {
more businesses embrace cloud com puting and microser­ await productRepository.AddProduct(product);
vices architecture, REST w ill play a crucial role in enabling return Results.Created
seamless com m unication between various services and ($"/addproduct/{product.Product ld }" , product);
systems. Furthermore, w ith the rise o f In te rn e t o f Things });
(loT ) devices and m obile applications, RESTful APIs w ill
be essential in fa c ilita tin g data exchange between these app.MapDelete("/deleteproduct/{id}" , async

codemag.com From SOAP to REST to GraphQL 67

 Listing 4: The IProductRepository interface

public interface IProductRepository }-

{ new Product {
public TaskcList<Product>> GetProducts(); Productjd = 3,
public Task<Product> GetProduct(int id); Product_Name = "DELL XPS Laptop",
} Description = "Dell XPS 9730 Laptop,
Intel Core i9 32GB 1TB SSD",
public class ProductRepository SKU = "DEL/19/1TB",
{ Quantity = 50,
private readonly List<Product> products = Price = 7500.00m
new List<Product> { }
new Product { };
Productjd = 1, public async TaskcList<Product>> GetProducts()
Product_Name = "HP Envy Laptop", {
SKU = "HPL/i9/lTB", return await Task.FromResult(products);
Description = }
"HP Envy Laptop i9 32 GB RAM 1 TB SSD", public async Task<Product> GetProduct(int id)
Quantity = 100, {
Price = 6500.00m return await Task.FromResult
}. (products.FirstOrDefault
new Product { (x => x.Productjd == id));
Productjd = 2, }
Product_Name = "Lenovo Legion Laptop", }
Description =
"Lenovo Legion Laptop i7 16 GB RAM 1 TB SSD",
SKU = "Len/i9/lTB/SSD",
Quantity = 150,
Price = 6000.00m

(int id, IProductRepository productRepository) => From REST to GraphQL

{ I t was Facebook's desire to fin d a method o f accessing
await productRepository.DeleteProduct(id); data th a t was both more e ffic ie n t and more elegant th a t
}); led to the developm ent o f GraphQL in the year 2012.
GraphQL achieves th is by adopting a declarative approach
Although the MapGet method has been used to create the toward retrieving and m anipulating data and enabling the
HttpGet endpoints, the MapPost method has been used to API clients to request only the data they require, thereby
create the HttpPost endpoint. Similarly, the MapDelete meth­ im proving performance and efficiency w hile reducing un­
od has been used here to create the HttpDelete endpoint. necessary data transfer. In order to retrieve data from
d iffe re n t endpoints using REST APIs, the c lie n t must make
Replace the con ten t o f the Program.es file w ith source m u ltip le requests, which can be in e ffic ie n t and tim e in ­
code given in Listing 5. You can now execute the ap­ tensive. I f your API is p ro ficie n t in retrieving a ll the data
plica tion and then use Postman to launch the endpoints. your application requires, you w o n 't encounter any issues
w ith over-fetching or under-fetching.

The prim ary goal o f GraphQL is to sim p lify the process

o f querying data by enabling clients to request exactly
the form at and structure o f data they need in a single
call. Remember, you need to make several calls to your
server to retrieve data s p lit across m ultiple data stores,
as shown in Figure 8.

Here are the key reasons why GraphQL is considered a b e t­

te r alternative to REST in some use cases.

• Data Retrieval: REST often necessitates accessing

various endpoints to colle ct d iffe re n t pieces o f in ­
form ation, p o te n tia lly leading to over-fetching or
under-fetching data. GraphQL enables clients to
specify the precise data they require from a single
endpoint, m inim izing the volume o f data tra n s m it­
ted across the network. Over fe tch in g is defined as a
situ a tio n in which an API returns more in fo rm atio n
than is necessary fo r your ap plication. For instance,
a c lie n t m ig h t request Order ID and Order Date and
receive Order Id , Order Date, and Product Id instead.
Under fe tch in g occurs when an API doesn't provide
a ll the data your application requests. A c lie n t may
DataBase
request Order Id and Order Date, b u t only receive Or­
Figure 8: A ty p ic a l GraphQL API A rchitecture der Id . As a consequence, the c lie n t may experience

68 From SOAP to REST to GraphQL codemag.com

 Listing 5: Configuring the REST endpoints in the Program.es file

using REST_Demo; Results.Ok(product) : Results.NotFoundQ);

var builder = WebApplication.CreateBuilder(args); app.MapPost("/addproduct" , async

(IProductRepository productRepository,
// Add services to the container. Product product) =>
{
builder.Services.AddScopedcIProductRepos itory, await productRepository.AddProduct(product);
ProductRepository>(); return Results.Created
var app = builder.BuildQ; ($"/addproduct/{product.Productjd}", product);
}):
// Configure the HTTP request pipeline.
app.MapDelete("/deleteproduct/{id}", async
app.MapGet("/getproducts" , async (int id, IProductRepository productRepository) =>
(IProductRepository productRepository) {
=> await productRepository.GetProducts()); await productRepository.DeleteProduct(id);
app.MapGet("/getproduct/{id:in t}" , async });
(IProductRepository productRepository, int id)
=> await productRepository.GetProduct(id) app.RunQ;
is Product product ?

reduced performance and im proper or in e ffic ie n t use and the types o f operations (queries and m utations)
o f memory, CPU, and network resources. th a t can be performed, thereby helping w ith valida­
• Versioning: Versioning in REST APIs manages chang­ tio n and introspection.
es to the APIs by assigning diffe ren t versions, such as • Improved performance: For applications th a t re­
v l, v2, and so on. GraphQL elim inates the necessity quire complex queries com bining m ultiple resources,
fo r version control by allowing clients to specify the GraphQL can be more e ffic ie n t than REST because i t
data they need in the query, making i t easier fo r APIs can gather a ll data in a single request.
to evolve w ith o u t breaking the existing queries. APIs
b u ilt w ith GraphQL do not require separate version­ There are certain downsides as w ell:
ing because clients or API consumers can define th e ir
requirements in the query and fetch the required data • Learning curve: Despite its simplicity, GraphQL is quite
w ith o u t breaking the existing queries. complex for those who are unfamiliar w ith its concepts.
• Type System: GraphQL employs a strongly typed The Learning curve for designing schemas, resolving
schema specifying the data form at you can request. queries, and securing GraphQL APIs can be quite steep.
This schema functions as a consensus between the • Caching challenges: Due to the dynamic nature o f
c lie n t and the server, thereby enabling the early de­ GraphQL queries, client-side and server-side caching
te ctio n o f errors. By recognizing p o te n tia l errors up can be more challenging compared to REST, where
fro n t, you can resolve the errors in a planned way URLs can easily serve as cache keys.
before they im p act your clients. • Increased complexity: GraphQL adds more complex­
ity to the server-side im plem entation in contrast to
conventional REST APIs. You should use resolvers to
Benefits and Downsides of GraphQL get the data you need. Managing complex queries
Here are the key benefits o f GraphQL: m ig h t in cur a d d itio n a l e ffo rt.
• Rate lim itin g : Im plem enting rate lim itin g in
• Efficient data querying: With GraphQL, clients can GraphQL is more complex than in REST because it's
query m ultiple resources and retrieve related data in harder to predict the cost o f a query due to its fle x ­
a single request. They can traverse the data graph ib le nature.
and retrieve only the required data, avoiding the • Security considerations: GraphQL APIs must be
over-fetching o f unnecessary fields or nested objects. carefully designed to avoid p o te n tia l vulne rab ilities.
• Reduced network traffic: GraphQL reduces the net­ Exposing to o much data or fu n c tio n a lity through the
work tra ffic and bandwidth consumption by m inim iz­ API can increase the attack surface, making proper
ing the payload size o f the responses. This explains au the ntica tion and authorization crucial.
why applications th a t Leverage GraphQL often e xh ib it
better performance compared to RESTful applications.
• Versioning and evolution: W ith GraphQL, depre­ GraphQL vs. REST
cated fields or types can be marked to signal clients A lthough REST and GraphQL are tw o o f the most popular
fo r m igration, allow ing fo r smooth API evolution approaches fo r bu ildin g APIs, there are subtle differences
w ith o u t breaking existing clients. between the tw o:
• Support fo r re al-tim e data: W ith GraphQL subscrip­
tions, clients can subscribe to specific data changes • Request form at: Each endp oint in REST specifies a
in real-tim e. Once subscribed, the clients are n o ti­ set o f resources and operations, and the c lie n t can
fied using events about any changes made to the typ ic a lly retrieve or m odify a ll resources using HTTP
data in real-tim e. methods, such as GET, POST, PUT, or DELETE. With
• Strongly typed schema: GraphQL enforces a robust GraphQL, clients request data based on a specific
typ in g system and a w ell-defined schema, providing structure th a t matches the server's schema.
c la rity fo r the available data types and fields. The • Data Retrieval: Each resource in REST can only
GraphQL schema defines the structure o f the data be accessed through a particular endpoint, mean-

codemag.com From SOAP to REST to GraphQL 69

 ing the client needs to make multiple requests to Request:
retrieve related data or complex object structures.
On the contrary, GraphQL enables the client to re­ GET /api/address?employee_id=l
trieve the exact data they need with a single query,
thereby reducing the number of requests required to Response:
fetch the data and minimizing network bandwidth
consumption. This can lead to fewer data transfers {
and more efficient API performance. "street": "Banjara Hills"
• Type System: In GraphQL, there's a strongly typed "city": "Hyderabad"
schema system th a t defines the types, fields, and "country": "India"
relationships between them. The client and server }
have a clear contract using this approach, and ad­
vanced tooling capabilities, such as schema intro ­ You can do the same in GraphQL in a much more elegant way.
spection and auto generation, are also available. On
the other hand, REST APIs do not typically include Request:
formal type systems, which makes them flexible but
also challenging to handle. query {
• Caching: REST enables you to cache frequently ac­ employee (id: 1) {
cessed data for faster access during subsequent calls id
to access the same piece of data, thereby reducing name
network traffic and improving application perfor­ address
mance. REST APIs can use HTTP caching techniques {
to minimize the data sent between clients and serv­ street
ers. Native caching mechanisms, on the other hand, city
are not supported by GraphQL APIs. A GraphQL API country
relies on client-side caching mechanisms to opti­ }
mize performance because query parameters may af­ }
fect the responses. }
• Protocol: Although REST works only with HTTP pro­
tocol, there are no protocol constraints in GraphQL.
In other words, GraphQL is agnostic of the transport Here's the GraphQL response to the preceding query:
layer— you can use i t with any transport layer pro­
tocol. {
• Partial Responses: GraphQL allows clients to retrieve "employee": {
specific information in a single query, minimizing "id": 1
data transmission. This is beneficial for sluggish net­ "name": "Joydip"
works or when accessing APIs via mobile apps. "address": {
"street": "Banjara Hills"
Applications with complex queries, a Large number of "city": "Hyderabad"
data sources, or unpredictable data needs may benefit "country": "India"
from GraphQL. REST may be more appropriate for simple }
CRUD applications when a mapping exists between the re­ }
sources and their endpoints. However, in REST, resources }
are typically returned in their entirety, which can lead to
over-fetching. Note th a t you could retrieve the entire data in ju s t one
call.

Comparing REST and GraphQL

Here's how REST and GraphQL compare against each oth­
Building a Simple Application
er in a typical request/response scenario. Consider two Using GraphQL
entities, Employee and Address. The former stores em­ It's time for writing some code. Let's now examine how to
ployee details and the Latter contains address details of build a simple ASP.NET Core 8 Web API application using
the employees. The following code snippets illustrate the GraphQL. A typical Order Processing System is comprised
requests and responses in REST to retrieve the details (in ­ of several entities such as Store, Supplier, Order, Product,
cluding address information) of an employee. Customer, etc. In this example, you'll implement only the
Store part of i t for simplicity.
Request:
Let's now examine how to create a ASP.NET Core 8 project
GET /api/employee?id=l in Visual Studio 2022.

Response: Create a NewASP.NETCore 8 Project in

Visual Studio 2022
{ You can create a project in Visual Studio 2022 in several
"id": 1 ways. When you Launch Visual Studio 2022, you'll see the
"name": "Joydip" Start window. You can choose "Continue w ithout code"
} to Launch the main screen of the Visual Studio 2022 IDE.

70 From SOAP to REST to GraphQL codemag.com

To create a new ASP.NET Core 8 Project in Visual Studio public string Email { get; set; }
2022: public string Phone { get; set; }
}
1. Start the Visual Studio 2022 IDE.
2. In the "Create a new project" window, select "ASP. The other entity classes are not being shown here for
NET Core Web API" and click Next to move on. brevity and also because this is a minimalistic implemen­
3. Specify the project name as GraphQL_Demo and the tation to illustrate how you can work with GraphQL in
path where i t should be created in the "Configure ASP.NET Core 7.
your new project" window.
4. I f you want the solution file and project to be cre­ Create the IStoreRepository Interface
ated in the same directory, you can optionally check Create a new .cs file named IStoreRepository in your proj­
the "Place solution and project in the same direc­ ect and replace the default generated code with the fo l­
tory" checkbox. Click Next to move on. lowing code snippet:
5. In the next screen, specify the target framework and
authentication type as well. Ensure th a t the "Con­ public interface IStoreRepository
figure for HTTPS," "Enable Docker Support," and the {
"Enable OpenAPI support" checkboxes are unchecked public Task<List<Store>> GetStoresQ;
because you won't use any of these in this example. public Task<Store> GetStore(int Id);
6. As you'll not leverage minimal APIs in this example, }
leave the Use controllers checkbox checked.
7. Click Create to complete the process. Create the StoreRepository Class
Next, create a new class named StoreRepository in a file
In this application, you'll take advantage of HotChocolate having the same name with a .cs extension. Now write the
to generate GraphQL schemas. With HotChocolate, you following code in there:
can build an extra layer on top of your application layer
th a t uses GraphQL. It's easy to set up and configure, and public class StoreRepository : IStoreRepository
i t eliminates the clutter of generating schemas. {

Install NuGet Package(s) }

So far so good. The next step is to install the neces­
sary NuGet Package(s). To install the required packages The StoreRepository class illustrated in the code snippet
into your project, right-click on the solution and the se­ below implements the methods of the IStoreRepository
lect Manage NuGet Packages for Solution.... Now search interface:
for the packages named HotChocolate.AspNetCore, and
HotChocolate.AspNetCore. Playground in the search box public async Task<List<Store>> GetStoresQ
and install them one after the other. Alternatively, you {
can type the commands shown below at the NuGet Package return await Task.FromResult(stores);
Manager Command Prompt: }

PM> Install-Package public async Task<Store> GetStore(int Id)

HotChocolate.AspNetCore {
PM> Install-Package return await Task.FromResult(stores.
HotChocolate.AspNetCore.Playground FirstOrDefault(x => x.Id == Id));
}
You can also install these packages by executing the fo l­
lowing commands at the Windows Shell: The complete source code of the StoreRepository class is
given in Listing 6.
dotnet add package
HotChocolate.AspNetCore Register the StoreRepository instance
dotnet add package The following code snippet illustrates how an instance of
HotChocolate.AspNetCore.Playground type IStoreRepository is added as a scoped service to the
IServiceCollection.
Create the Model Class
Create a new class named Order in a file having the same builder.Services.AddScoped
name with a .cs extension and write the following code IStoreRepository, StoreRepository>();
in there:
Create the GraphQL Query Class
public class Store A GraphQL query is defined as a request sent by the cli­
{ ent to the server. In GraphQL, clients request data from
public int Id { get; set; } the server using queries th a t adhere to a specific struc­
public string Name { get; set; } ture and syntax per the GraphQL specification. When us­
public string Address { get; set; } ing GraphQL queries, clients may specify data and the
public string City { get; set; } response format.
public string State { get; set; }
public string Country { get; set; } There are several fields in the query th a t represent the
public string Zip { get; set; } data th a t was retrieved from the API. The data contained

codemag.com From SOAP to REST to GraphQL 71

 Listing 6: The StoreRepository class

public class StoreRepository : IStoreRepository Email = "[email protected]"

{ },
private readonly List<Store> stores = new Store
new List<Store> {
{ Id = 3,
new Store Name = "Harrods",
{ Address = "Flat 60 Davis Road",
Id = 1, City = "Bradford",
Name = "Walmart", State = "Yorkshire",
Address = "274 Reagan Apt. 919", Country = "UK",
City = "Huntington", Zip = "BD1 1BL",
State = "West Virginia", Phone = "1111111111",
Country = "USA", Email = "[email protected]"
Zip = "25049", }
Phone = "1111111111", };
Email = "[email protected]" public async Task<List<Store» GetStoresO
}, {
new Store return await Task.FromResult(stores);
{ }
Id = 2, public async Task<Store> GetStore(int Id)
Name = "Amazon", {
Address = "57526 Michelle Ferry Suite 714", return await Task.FromResult(
City = "Edmond", stores.FirstOrDefault(x => x.Id == Id));
State = "Oklahoma", }
Country = "USA",
Zip = "66347",
Phone = "1111111111",

Listing 7: The StoreQuery class

using HotChocolate.Subscriptions; {
List<Store> stores =
namespace GraphQL_Demo await StoreRepository.GetStoresO;
{ await eventsender.SendAsync
public class StoreQuery ("Returned a List of Stores", stores);
{ return stores;
public async Task<List<Store>> }
GetAllStores( [Service] }
IStoreRepository StoreRepository, }
[Service] ITopicEventSender eventsender)

Listing 8: The StoreType class

using HotChocolate.Types; s.Address).Type<StringType>();

descriptor.Field(s =>
namespace GraphQL_Demo s.City).Type<Stri ngType>();
{ descriptor.Field(s =>
public class StoreType : s.State).Type<StringType>();
ObjectType<Store> descriptor.Field(s =>
{ s.Country).Type<Stri ngType>();
protected override void Configure descriptor.Field(s =>
(lObj ectTypeDescri ptor s.Zip).Type<StringType>();
<Store> descriptor) descriptor.Field(s =>
{ s.Email),Type<StringType>();
descriptor.Field descriptor.Field(s =>
(s => s.Id).Type<IdType>(); s.Phone).Type<StringType>();
descriptor.Field(s =>
s.Name).Type<Stri ngType>();
descriptor.Field(s =>

in each o f these fields can be traversed and retrieved by Create a GraphQL Subscription
nesting them . Create a new .cs file named StoreQuery in You should also create a subscription to enable your
your project and replace the de fau lt generated code w ith GraphQL server to n o tify a ll subscribed clients when an
the code given in Listing 7. event occurs. Create a new class named StoreSubscription
and replace th e de fau lt generated code w ith the source
Create the GraphQL Object Type code given in Listing 9.
In GraphQL, Object Types are used to describe the type of data
fetched using your API and they are represented by creating Configure GraphQL Server in ASP.NETCore
a class th a t derives the GraphQLJypes.ObjectGraphType class. Once you've created the Query type to expose the data
Create a new file named StoreType.es in your project and re­ you need, you should configure GraphQL Server in the
place the default code w ith the code given in Listing 8. Program.es file using th e fo llo w in g code snippet:

72 From SOAP to REST to GraphQL codemag.com

 x O X

O D focalhost

S
C
O
D

A
M
E
H
C
S

Figure 9: The storeB yld query in execution!

builder.Services.AddGraphQLServer() Listing 9: The StoreSubscription class

.AddType<StoreType>() using HotChocolate.Execution;
.AddQueryType<StoreQuery>() using HotChocolate.Subscriptions;
.AddSubscriptionType<StoreSubscription> ()
.AddlnMemorySubscriptions(); namespace GraphQL_Demo
{
public class StoreSubscription
You can then call th e MapGraphQL method to register the {
middleware: [SubscribeAndResolve]
public async ValueTask<ISourceStream
<List<Store>>> OnStoreGet( [Service]
app. MapGraphQLO;
ITopicEventReceiver eventReceiver,
CancellationToken cancellationToken)
When you register th is middleware, the GraphQL server w ill {
be available a t /g ra p h q l by default. You can also customize return await eventReceiver.SubscribeAsync
the endpoint where the GraphQL server w ill be hosted by
<List<Store»
("Returned Stores", cancellationToken);
specifying the follow ing code in the Program.es file :
}
}
app.MapGraphQL("/graphql/mycustomendpoi nt"); }

Listing 10: Configuring GraphQL in th e Program.es file

using GraphQL_Demo;
using HotChocolate.AspNetCore; var app = builder.BuildQ;
using HotChocolate.AspNetCore.Playground;
11 Configure the HTTP request pipeline.
var builder = WebApplication.CreateBuiIder(args);
app.UseAuthorization();
// Add services to the container.
builder.Services.AddScoped app.MapControllers();
<IStoreRepository, StoreRepository>(); app.UsePlayground(new PlaygroundOptions
{
builder.Services.AddGraphQLServer() QueryPath = "/graphql",
.AddType<StoreType>() Path = "/playground"
.AddQueryType<StoreQuery>() });
.AddSubscriptionType<StoreSubscription>()
.AddlnMemorySubscriptions(); app.MapGraphQLO;
app.Run();
builder.Services.AddControllers();

codemag.com From SOAP to REST to GraphQL 73

 CODE COMPILERS

Listing 11: The StoreControLLer class

using Microsoft.AspNetCore.Mvc;
namespace GraphQL_Demo.Controllers
{
CODE^Q
May/Jun 2024
Volume 25 Issue 3
YEARS

[Route("api/[controller]")]
[ApiController]
public class Storecontroller : ControllerBase Group Publisher
Markus Egger
{
private IStoreRepository _storeRepository; Editor-in-Chief
public Storecontroller Rod Paddock
(IStoreRepository storeRepository)
Managing Editor
{
_storeRepository = storeRepository; Ellen W hitney

} Content Editor
Melanie Spiller
[HttpGet("{id}")]
public async Task<Store> GetStore(int id) Writers in This Issue
Markus Egger Kevin Goff
{
return await _storeRepository.GetStore(id); Joydip Kanjilal Julie Lerman
Sahil Malik Mike Rousos
}
Paul D. S heriff
[HttpGet("GetStores")] Technical Reviewers
public async Task<List<Store» GetStoresO Markus Egger
{ Rod Paddock
return await _storeRepository.GetStoresO;
Production
}
Friedl Raffeiner Grafik Studio
}
w w w .frigraf.it
}
Graphic Layout
Friedl Raffeiner Grafik Studio in collaboration
w ith onsight (www .onsightdesign.info)
Listing 10 shows the complete source of the Program.es file.
Printing
Fry Communications, Inc.
Now execute the application and browse the /playground 800 West Church Rd.
endpoint. Next, execute the following query: Mechanicsburg, PA 17055

Advertising Sales
query Tammy Ferguson
832-717-4445 ext. 26
{ [email protected]
storeByld (id: 1)
Circulation & Distribution
{ General Circulation: EPS Software Corp.
id Newsstand: Ingram Periodicals, Inc.
name In te rn a tio n a l Bonded Couriers (IBC)
Media Solutions
address Source In te rlin k In te rn a tio n a l
}
Subscriptions
}
Circulation M an ag er
Colleen Cade
Figure 9 shows the output on execution of the application. 832-717-4445 ext. 28
[email protected]
Create the StoreController Class
US subscriptions are $29.99 USD fo r one year.
Finally, you need to build the controller class to expose Subscriptions outside the US are $50.99 USD.
the endpoints to the outside world so th a t they can be Payments should be made in US dollars drawn
consumed by the authenticated clients or consumers of on a US bank. American Express, MasterCard,
Visa and Discover credit cards accepted.
the API. To do this, create a new API Controller in your Back issues are available. For subscription
project named StoreController and write the code given in inform ation, email subscriptions@ code-magazine.com
Listing 11 in there. or contact customer service a t 832-717-4445 ext. 9.

Subscribe online at
www.code-magazine.com
Conclusion
CODE Developer Magazine
The requirements and constraints of your project w ill de­ EPS Software Corporation / Publishing Division
termine the most appropriate choice between SOAP, REST, 6605 Cypresswood Drive, Ste 425, Spring, Texas 77379 USA
Phone: 832-717-4445
or GraphQL. SOAP is a good choice for enterprise-level
applications because of better support for security and
its robust contract definitions. REST can significantly
enhance a resource-oriented application because of bet­
ter performance, enhanced scalability, and simplicity.
GraphQL offers a flexible, efficient approach to data re­
trieval and manipulation, making i t an excellent choice
for applications th a t require strong typing, real-time data
updates, and efficient data retrieval capabilities.

Joydip Kanjilal
CODE

74 From SOAP to REST to GraphQL

 CUSTOM SOFTWARE DEVELOPMENT
STAFFING TRAINING/MENTORING SECURITY

MORE THAN JUST

A MAGAZINE!
Does your developm ent team lack skills or tim e to com plete all your business-critical software projects?
CODE Consulting has to p -tie r developers available w ith in-depth experience in .NET,
web developm ent, desktop developm ent (WPF), Blazor, Azure, m obile apps, loT and more.

CONTACT US TODAY FOR A COMPLIMENTARY ONE HOUR TECH CONSULTATION.

k
NO STRINGS. NO COMMITMENT. JUST CODE.
c
o
t
sr
e
tt
u
codemag.com/code
h
s
© 832-717-4445 ext. 9 • [email protected]
CODE STAFFING

UNLOCK ±
STAFFING ~
EXCELLENCE
Top-Notch IT Talent, Contract Flexibility, Happy Teams, and a
Commitment to Customer Success Converge with CODE Staffing

Our IT staffing solutions are engineered to drive your business forward while
saving you time and money. Say goodbye to excessive overhead costs and
lengthy recruitment efforts. With CODE Staffing, you'll benefit from contract
flexibility that caters to both project-based and permanent placements. We
optimize your workforce strategy, ensuring a perfect fit for every role and
helping you achieve continued operational excellence.

Ready to Discuss Your IT Staffing Needs?

Visit our website to find out more about how we are changing
the staffing industry.

Website: codestaffing.com
Yair Alan Griver (yag)
Chief Executive Officer
Direct: +1 425 301 1590
Email: [email protected]