TheoryTofTAutomata

AssignmentT#T1
BY
M.TSuhaibTKhalid
RollTno:T105
BSCST(4thTSemester)
Section:TC

SubmittedTto
Sir.TUmarTFarooq
 Language
IfTΣTisTalphabet,TandTL⊆Σ∗,TthenTLisa(formal)TlanguageToverTΣ.
AT(possiblyTinfinite)TsetTofTstringsTallTofTwhichTwereTchosenTfromTsomeT
Σ∗.
ATlanguageToverTΣTneedTnotTincludeTstringsTwithTallTsymbolTofTΣ.TThus,
TaTlanguageToverTΣTareTalsoTaTlanguageToverTanyTalphabetTthatTisTaTsup
ersetTofTΣ.
ATlanguageTisTaTsetTofTstrings.TOneTspecialTlanguageTisTΣ∗,TwhichTisTthe
TsetTofTallTpossibleTstringsTgeneratedToverTtheTalphabetTΣ∗.TForTexample,
Tif
ΣT=T{a,b,c}TthenTΣ∗T=T{a,b,c,aa,ab,ac,ba,...,aaaaaabbbaababa,...}
Namely,TΣ∗TisTtheT“full”TlanguageTmadeTofTcharactersTofTΣ.TNaturally,Tan
yTlanguageToverTΣTisTgoingTtoTbeTaTsubsetTofTΣ∗.
Examples:
1.TProgrammingTlanguageTC
2.TLegalTprogramsTareTaTsubsetTofTtheTpossibleTstringsTthatTcanTbeTforme
dTfromTtheTalphabetTofTtheTlanguageT(aTsubsetTofTASCIITcharacters).
OtherTLanguageTExamples:
1.TGT=T{a,ab,aab,aaab,aaaab,...}
2.TGT=T{bb,ab,ba}
3.TΣ∗TisTaTlanguageTforTanyTalphabetTΣ.
4.T∅,TtheTemptyTlanguage,TisTaTlanguageToverTanyTalphabet.
5.TTheTlanguageTofTallTstringsTconsistingTofTnT0’sTfollowedTbyTnT1’sT(n≥
0):
{00011,0000111,000001111,...}
6.TTheTsetTofTstringsTofT0’sTandT1’sTwithTanTequalTnumberTofTeach:
{0110,0110,000111,010011,1001,...}
7.T{TaT|TaTcontainTanTequalTnumberTofT0’sTand1’sT}
 LexicographicTordering
LexicographicTorderingTofTaTsetTofTstringsThereTanTorderingTofTstringsTtha
tThaveTshorterTstringsTfirstTandTsortTtheTstringsTalphabeticallyTwithinTeachT
length.TNaturally,TweTassumeTthatTweThaveTanTorderTonTtheTgivenTalphabe
t.T
Example:
ForTΣT=T{a,b},TtheTLexicographicTorderingTofTΣ∗TisTa,b,aa,ab,baa,bbb,aaaaa
,aaab,....

LanguagesTandTSetTNotation
MostTofTtheTtimeTitTwouldTbeTmoreTusefulTtoTuseTsetTnotationsTtoTdefine
TaTlanguage;TthatTis,TdefineTaTlanguageTbyTtheTpropertyTtheTstringsTinTthi
sTlanguageTpossess.
Example:
ConsiderTtheTfollowingTsetTofTstrings.
LT=T{zT|TzT∈T{a,b}*TandT|z|TisTevenT}

ImportantToperatorsTonTlanguages
Union
TheTunionTofTtwoTlanguagesTLTandTM,TdenotedTL∪M,TisTtheTsetTofTstrin
gsTthatTareTinTeitherTL,TorTM,TorTboth.
Examples:
IfTGT=T{0001,100,1111}TandTNT=T{0001}Tthen,
L∪M={0001,100,1111}
 Concatenation
TheTconcatenationTofTlanguagesTLTandTM,TdenotedTL.MTorTjustTLM,TisTt
heTsetTofTstringsTthatTcanTbeTformedTbyTtakingTanyTstringTinTLTandTconc
atenatingTitTwithTanyTstringTinTM.

Examples:
IfTLT=T{0001,100,1111}TandTMT=T{0001}Tthen,T
L.MT=T{0001,100,1111,001001,10001,111001}

Closure
TheTclosureTofTaTlanguageTLTisTdenotedTasTL∗TandTrepresentsTtheTsetTof
TthoseTstringsTthatTcanTbeTformedTbyTtakingTanyTnumberTofTstringsTfrom
TL,TpossiblyTwithTrepetitionsT(i.e.,TtheTsameTstringTmayTbeTselectedTmore
TthanTonce)TandTconcatenatingTallTofTthem.
Examples:
1.TIfTLT=T{0,1}TthenTL∗TareTallTstringsTofT0TandT1.T
2.TIfTLT=T{0,11}TthenTL∗TconsistsTofTstringsTofT0’sTandT1’sTsuchTthatTth
eT1comeTinTpairs,Te.g.,T011,T11110TandTǫ.TButTnotT01011TorT101.

LexicalTAnalyzer
1.TTheTfirstTphaseTofTaTcompiler.T
2.TLexicalTanalysis:TprocessTofTtakingTanTinputTstringTofTcharactersT(such
TasTtheTsourceTcodeTofTaTcomputerTprogram)TandTproducingTaTsequenceTo
fTsymbolsTcalledTlexicalTtokens,TorTjustTtokens,TwhichTmayTbeThandledTm
oreTeasilyTbyTaTparser.
3.TTheTlexicalTanalyzerTreadsTtheTsourceTtextTand,Tthus,TitTmayTperformTc
ertainTsecondaryTtasks:
EliminateTcommentsTandTwhiteTspacesTinTtheTformTofTblanks,TtabTandTne
wTlineTcharacters.
CorrelateTerrorsTmessagesTfromTtheTcompilerTwithTtheTsourceTprogramT(Te.
g,TkeepTtrackTofTtheTnumberTofTlines).
4.TTheTinteractionTwithTtheTparserTisTusuallyTdoneTbyTmakingTtheTlexicalT
analyzerTbeTaTsub-routineTofTtheTparser.

Tokens,TPatterns,TLexemes
1.TToken:TATtokenTisTaTgroupTofTcharactersThavingTcollectiveTmeaning:Tt
ypically,TaTwordTorTpunctuationTmark,TseparatedTbyTaTlexicalTanalyzerTand
TpassedTtoTaTparser.
2.TATlexemeTisTanTactualTcharacterTsequenceTformingTaTspecificTinstanceT
ofTaTtoken,TsuchTasTnum.
3.TPattern:TATruleTthatTdescribesTtheTsetTofTstringsTassociatedTtoTaTtoken
.TExpressedTasTaTregularTexpressionTandTdescribingThowTaTparticularTtoken
TcanTbeTformed.TForTexample,
[A-Za-z]T[A-Za-z_0-9]T*
TheTpatternTmatchesTeachTstringTinTtheTset.
ATlexemeTisTaTsequenceTofTcharactersTinTtheTsourceTtextTthatTisTmatched
TbyTtheTpatternTforTaTtoken.

Example:

TokenTSampleTLexemesT(Informal)TDescriptionTofTPattern
TConstTconstTconst
TIfTifTif
TRelationT<,<=,=,<>,>,=>T<|<=|=|<>|>|=>T
TIdTpi,Tcount,TD2T(letter.(letterT|Tdigit)∗)
TNumT3.1426,T0.6,T6.22TanyTnumericTconstant
LiteralT”coreTdumped”TanyTcharacterTbetweenT”and”Texcept”
 LexicalTErrors
1.TFewTerrorsTareTdiscernibleTatTtheTlexicalTlevelTalone.
2.TLexicalTanalyzerThasTaTveryTlocalizedTviewTofTtheTsourceTtext.
3.TItTcannotTtellTwhetherTaTstringTfiTisTamTisTspellingTofTaTkeywordTifTo
rTanTidentifier.
4.TTheTlexicalTanalyzerTcanTdetectTcharactersTthatTareTnotTinTtheTalphabet
TorTstringsTthatThaveTnoTpattern.
5.TInTgeneral,TwhenTanTerrorTwereTfound,TtheTlexicalTanalyzerTstopsT(butT
otherTactionsTareTalsoTpossible).

StagesTofTaTlexicalTanalyzer
Scanner
1.TBasedTonTaTfiniteTstateTmachine.
2.TIfTitTlandsTonTanTacceptingTstate,TItTtakesTnoteTofTtheTtypeTandTpositi
onTofTtheTacceptance,TandTcontinues.
3.TSometimesTitTlandsTonTaT"deadTstate,T"whichTisTaTnon-acceptingTstate.
4.TWhenTtheTlexicalTanalyzerTlandsTonTtheTdeadTstate,TitTisTdone.TTheTlas
tTacceptingTstateTisTtheToneTthatTrepresentTtheTtypeTandTlengthTofTtheTlon
gestTvalidTlexeme.
5.TTheT"extra"TnonTvalidTcharacterTshouldTbeT"returned"TtoTtheTinputTbuff
er.

Evaluator
1.GoesToverTtheTcharactersTofTtheTlexemeTtoTproduceTaTvalue.
2.TheTlexeme’sTtypeTcombinedTwithTitsTvalueTisTwhatTproperlyTconstitutes
TaTtoken,TwhichTcanTbeTgivenTtoTaTparser.
3.SomeTtokensTsuchTasTparenthesesTdoTnotTreallyThaveTvalues,TandTsoTthe
TevaluatorTfunctionTforTtheseTcanTreturnTnothing.
4.TheTevaluatorsTforintegers,Tidentifiers,TandTstringsTcanTbeTconsiderablyTm
oreTcomplex.
5.SometimesTevaluatorsTcanTsuppressaTlexemeTentirely,TconcealingTitTfromTt
heTparser,TwhichTisTusefulTforTwhiteTspaceTandTcomments.

ImplementationTofTLexicalTAnalyze
DifferentTwaysTofTcreatingTaTlexicalTanalyzer:
1.TToTuseTanTautomaticTgeneratorTofTlexicalTanalyzersT(asTLEXTorTFLEX)
.
2.TThoughTitTisTpossibleTandTsometimesTnecessaryTtoTwriteTaTlexerTbyTha
nd,TlexersTareTofTtenTgeneratedTbyTautomatedTtools.
3.TTheseTtoolsTacceptTregularTexpressionsTwhichTdescribeTtheTtokensTallow
edTinTtheTinputTstream.TInput:TSpecification.
4.TRegularTexpressionsTrepresentingTtheTpatterns.
5.TActionsTtoTtakeTaccordingTtoTtheTdetectedTtoken.
6.EachTregularTexpressionTisTassociatedTwithTaTphraseTinTaTprogrammingTl
anguageTwhichTwillTevaluateTtheTlexemesTthatTmatchTtheTregularTexpressio
n.
7.TTheTtoolTthenTconstructsTaTstateTtableTforTtheTappropriateTfiniteTstateTm
achineTandTcreatesTprogramTcodeTwhichTcontainsTtheTtable,TtheTevaluationT
phrases,TandTaTroutineTwhichTusesTtheTmTappropriately

AdvantagesT&TDisadvantages
LexicalTanalyzerTgenerator
Advantages:TeasierTandTfasterTdevelopment.
Disadvantages:TtheTlexicalTanalyzerTisTnotTveryTefficientTandTitsTmaintenan
ceTcanTbeTcomplicated.
ToTwriteTtheTlexicalTanalyzerTbyTusingTaThigh-levelTlanguage
Advantages:TMoreTefficientTandTcompact.
Disadvantages:TDoneTbyThand.
ToTwriteTtheTlexicalTanalyzerTbyTusingTaTlow-levelTlanguage
Advantages:TVeryTefficientTandTcompact.
Disadvantages:TDevelopmentTisTcomplicate.

SyntaxTAnalyzer

ParsingT=TdeterminingTwhetherTtheTstringTofTtokensTcanTbeTgeneratedTbyT
aTgrammar

Grammars
1.TPrecise,Teasy-toTunderstandTdescriptionTofTsyntax.
2.TContext-freeTgrammarsT->TefficientTparsersT(automatically!).T
3.THelpTinTtranslationTandTerrorTdetection.
4.TE.g.TAttributeTgrammars.
5.TEasierTlanguageTevolution.T
6.TCanTaddTnewTconstructsTsystematically.

SyntaxTErrors
1.TManyTerrorsTareTsyntacticTorTexposedTbyTparsing.T
2.TE.g.TUnbalancedT().T
3.TErrorThandlingTgoals:T
4.TReportTerrorsTquicklyT&Taccurately.T
5.TRecoverTquicklyT(continueTparsingTafterTerror).T
6.TLittleToverheadTonTparseTtime.

ErrorTRecovery
PanicTmode
1.TDiscardTtokensTuntilTsynchronizationTtokenTfoundT(oftenT‘;’).
PhraseTlevelT
1.TLocalTcorrection:TreplaceTaTtokenTbyTanotherTandTcontinue.
ErrorTproductionsT
1.TEncodeTcommonlyTexpectedTerrorsTinTgrammar.T
GlobalTcorrectionT
1.TFindTclosestTinputTstringTthatTisTinTL(G).T
2.TTooTcostlyTinTpractice.

Context-freeTGrammars
1.TPreciseTandTeasyTwayTtoTspecifyTtheTsyntacticalTstructureTofTaTprogram
mingTlanguage.T
2.TEfficientTrecognitionTmethodsTexist.
3.TNaturalTspecificationTofTmanyT“recursive”Tconstructs:T
4.TexprT->TexprT+TexprT|Tterm.

ShortThandsT&TDerivations
1.TET->TET+TET|TET*TET|T(E)T|T-TET|T<id>
2.TET=>T-TET“ETderivesT-E”T
3.T=>TderivesTinT1TstepT
4.T=>T*TderiveTinTnT(0..)Tsteps

MoreTDefinitions
1.TL(G)TlanguageTgeneratedTbyTGT=TsetTofTstringsTderivedTfromTS.T
2.TST=>+TwT:TwTsentenceTofTGT(wTstringTofTterminals).
3.TST=>+TαT:TαTsententialTformTofTG.T
4.T(stringTcanTcontainTnonterminals).T
5.TGTandTG’TareTequivalentT:⇔TL(G)T=TL(G’).T
6.TATlanguageTgeneratedTbyTaTgrammarT(ofTtheTformTshown)TisTcalledTaT
context-freeTlanguage.

Example:
GT=T({-,*,(,),<id>},T{E},TE,T{ET->TET+TE,TE->TET*TET,TET->T(E)T,TE-
>T-TE,TET->T<id>})

Sentence:T-(<id>T+T<id>)TDerivation:
ET=>T-ET=>T-(E)T=>T-(E+E)T=>T-(<id>+E)T=>T-(<id>T+T<id>)