Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

Access Tech n iqu es: Fast Table

Look u p Fu n ct ion s
Awarded Article by harfang On 2009-12-21

Views: 12,091 24,698 Points

In t r odu ct ion

In database applications, it's often useful to retr ieve a

single item of inform ation from a table. For exam ple the
full nam e of a com pany from its ID num ber, the current
price of a product, or the num ber of visits of a patient.
When this is needed in a query, sever al SQL techniques
are available to obtain and use this infor m ation. In the
inter face or fr om Visual Basic for Access, dom ain lookup
functions are used: DLookup, DSum , DCount, etc.

The present article shows that DLookup is versatile, but

com par atively slow. If the area of application is restricted
and focused on very sim ple table lookups, it becom es
possible to wr ite functions with m uch better
perform ance. However that possibility is subject to the
following constraints:

» The table is in a Jet (Access) database.

» The search doesn't involve wild-cards or any
calculations.
» The Úeld(s) in the criteria ar e (or should be) indexed.

If these conditions ar e m et, a single item of inform ation

can be retr ieved from a table up to 100 tim es faster than
by using DLookup. This sor t of speed opens up new
possibilities, and o˜ ers solutions to problem s typically
considered too com plex for Access.

Incidentally, the Úrst sections show how plain DLookup

can be im plem ented in stand alone Visual Basic or in
VB.NET -- for that m atter in any pr ogr am using the Jet
Engine as database m anager -- when Access built-in
functions are not available. The other dom ain functions
can then be sim ulated easily as well. This idea is
im plem ented in a com panion article.

DLook u p syn t ax

The help Úle provides: DLookup( expr, dom ain [, criteria>

])

The thr ee argum ents ar e in fact three SQL clauses, and

the sam e infor m ation is obtained by running the
1 de 9 05/07/2016 01:09 p.m.
Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

The thr ee argum ents ar e in fact three SQL clauses, and

the sam e infor m ation is obtained by running the
following quer y:

SELECT [Top 1] expr

FROM dom ain
WHERE criteria

The "Top 1" restriction is neither necessary nor actually

included, but only im plied since DLookup returns a single
value. The expression argum ent can really be any legal
deÚnition of a Úeld (without the aliased Úeld nam e), and
the criteria can contain subqueries. Both requir e square
brackets when the Úeld nam e is im pr oper, unlike the
dom ain.

The syntax for the other dom ain lookup functions is

alm ost identical, they sim ply wrap the expression into one
of the aggr egate functions of SQL. They can in fact all be
sim ulated by DLookup:

DLookup('Count(* )', 'Em ployees')

DLookup('Sum (Quantity)', 'Order Details', 'ProductID =
12')
DLookup('Avg([Unit Pr ice])', 'Products')

We are not going to writer faster versions for aggr egate

functions here; this article focuses on looking up single
Úeld values as quickly as possible. However, there are fast
function sim ilar to DSum (), DMax() and DMin() in the
second dem o, along with som e other s that are not
available in Access -- and that m ight have been called
DNext(), DPrev(), and DList() if they were.

Sear ch in g for fast er look u p t ech n iqu es

If you are looking for the fastest version only, this section
can be skipped without har m ; it descr ibes in steps how
speed is achieved by reducing ver satility. The Úrst attem pt
is quite obvious.
1
Function MyDLookup(Expr As String, Domain As String, Optional Cri

Dim strSQL As String

strSQL _
= " SELECT " & Expr _
& " FROM " & Domain _
& " WHERE " + Criteria

With CurrentDb.OpenRecordset(strSQL, dbOpenDynaset)

If .RecordCount Then MyDLookup = .Fields(0) Else MyDLooku
End With

End Function

Open in new window

This alm ost em ulates DLookup. The '+' concatenation

operator ensures that the entir e WHERE clause is nulled
2 de 9 when no cr iter ia is pr ovided. The query is opened as 05/07/2016 01:09 p.m.
Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

This alm ost em ulates DLookup. The '+' concatenation

operator ensures that the entir e WHERE clause is nulled
when no cr iter ia is pr ovided. The query is opened as
r ecordset, and the Úr st Úeld, the expr ession is r eturned if
and only if a record is retrieved.

Benchm ar king this function shows that it's noticeably

slower than the built-in version. Ther e m ust be a
problem . The function is tight and "by the book", but the
experienced developer will quickly suspect the call to
Curr entDb (on line 10).

In all current versions of Access since 97 at least,

Curr entDb doesn't retur n the curr ent database object.
Instead it's a new object with all collections r efreshed. I
will forgo the discussion of reasons and consequences of
this behaviour , but point out that it's slow and
unnecessary, except -som etim es- when tables or queries
are being created and deleted.

¯¯¯¯¯
If we use an existing database object like DBEngine(0)(0),
then the function will triple in speed and becom e alm ost
50% faster than DLookup.
2
With DBEngine(0)(0).OpenRecordset(strSQL, dbOpenDynaset)
' or, if an open database object exists:
With MyDb.OpenRecordset(strSQL, dbOpenDynaset)

Open in new window

It seem s odd that DLookup() is outper for m ed so easily.

Mor eover , this im plem entation provides som e record
sour ce Ûexibility, because the MyDLookup dom ain
argum ent can accept a m ulti-table Join clause, which
DLookup r ejects. [Note: this isn't r eally a great
im provem ent, especially as the cr iter ia argum ent of both
functions accept sub-queries.]

It m ust be that DLookup perfor m s som e additional tasks

to im pr ove robustness r ather than speed, like general
error m anagem ent and recovery: ver ifying that dom ain is
a table or a query, adding square brackets if needed,
r efreshing both collections for a second attem pt if
dom ain was not found... It also runs in a di˜ erent nam e
space, but that's irrelevant at the m om ent.

¯¯¯¯¯
By using the sam e technique as DLookup, we achieved an
unexpected sm all im provem ent. Let's try another
approach: opening a recordset and using the .FindFirst
m ethod. Let's also sim plify the problem by looking for a
Úeld and not an expr ession.
3
Function MyDLookup(Field As String, Domain As String, Criteria As

Dim rec As DAO.Recordset

Set rec = DBEngine(0)(0).OpenRecordset(Domain, dbOpenDynaset)

rec.FindFirst Criteria
If Not rec.NoMatch Then MyDLookup = rec(Field)
3 de 9 05/07/2016 01:09 p.m.
Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

Set rec = DBEngine(0)(0).OpenRecordset(Domain, dbOpenDynaset)

rec.FindFirst Criteria
If Not rec.NoMatch Then MyDLookup = rec(Field)

End Function

Open in new window

Benchm ar king this function shows som ething quite

di˜ erent: the r elative execution tim e rises very quickly
with the table size being searched. It is slightly faster than
DLookup on tiny tables (100 recor ds), but dr am atically
slower on any ser ious data, even when the Úeld in the
criteria is indexed.

And that's our Úrst lead. What happens when running a

select quer y com par ed to using the "Únd Úrst" m ethod?
Jet optim izes the query before even looking at the Úrst
r ecord. The recor dset's m ethod, on the other hand,
doesn't use an index even if it is available.

¯¯¯¯¯
We can't close this exploration without tr ying param etric
queries. This approach is logical for program m ers
switching from m ost other SQL engines: trying to em ulate
stor ed procedures. This m ethod is in fact recom m ended
for "pass-through" quer ies, not very useful for Jet
databases, but let's try.

For the test to be valid, the query deÚnition should be

kept open between calls to the function, which m eans it
cannot be generic. Instead, this function specialises in a
single task.
4
Function CompanyName(Key)

Static qdf As QueryDef

Dim strSQL As String

If qdf Is Nothing Then

strSQL _
= " SELECT CompanyName" _
& " FROM Customers" _
& " WHERE CustomerID = ?"

Set qdf = CurrentDb.CreateQueryDef("", strSQL)

End If

With qdf
.Parameters(0) = Key
With .OpenRecordset
If .EOF _
Then CompanyNameQdf = Null _
Else CompanyNameQdf = .Fields(0)
End With
End With

End Function

Open in new window

When tested, the function perform s m uch like the

function (2), 'MyDLookup', without the versatility.
Although pass-though quer ies can be very eı cient in
other circum stances, they do not seem useful for Jet
databases, as could be expected.

Many of the functions presented up to here leave r oom

4 de 9 05/07/2016 01:09 p.m.
Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

Many of the functions presented up to here leave r oom

for im provem ent, especially by using m ore static objects
and by optim ising for speciÚc scenarios (looking in the
sam e table m ost of the tim e, searching in a speciÚc key
order, calling the function from a program loop allowing
very speciÚc assum ptions, etc.). If you use any of them , do
spend som e tim e optim ising. In this article, however, we
keep looking for yet another solution.

Ou t per for m in g DLook u p

Despite som e noticeable im provem ents, the functions

above all play in the sam e leage. To Únd the highly
specialised tool, perform ing one task only, but r ight, in the
least am ount of tim e, we need two things: lower level
techniques and m ore inform ation about the tar get.

We can guess that all lookup techniques that rem ain

stable with regard to table size use an index. We know an
index exists because we perfor m ed all lookups on the key
Úeld of the table. Let's form alise this r estriction: we ar e
now looking for a Úeld in a table, and the criteria m ust be
deÚned on an index. Let's also try to use the index
explicitly.

An Index property exists for recor dsets, cor related to the

Seek m ethod, but only for "table-type" recor dsets. This
m eans opening the table directly, without any inter face
layer. When one double-clicks "Products", one doesn't
"open the table"; instead the quer y "TABLE Products" is
opened, shorthand for "SELECT * FROM Products". The
data displayed is in fact a Dynaset -- a dynam ic set of
r ecords freshly read from the table. A Dynaset creates a
num ber of illusions for the beneÚt of the user: each
r ecord is assigned a row num ber, stable until the
r ecordset is refreshed; ther e is an actual row count;
sear ching, Últering and sorting is possible on any Úeld;
etc.

Raw tables -- r ecordsets opened with the "open table"

option -- do not even have a recor d count. You would
need to create a loop to count them ! The recor ds are
unsorted, unless you choose one of the indexes, m aking
them appear to be ordered. If an index is selected, you
can navigate the index (next, previous) and also seek a
key value.

That's prom ising.

5
Function DSeek( _
Field As String, Table As String, Index As String, Key1, _
Optional Key2, Optional Key3, Optional Key4, Optional Key5 _
) As Variant

With DBEngine(0)(0)(Table).OpenRecordset(dbOpenTable)
.Index = Index
5 de 9 .Seek "=", Key1, Key2, Key3, Key4, Key5 05/07/2016 01:09 p.m.
Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

) As Variant

With DBEngine(0)(0)(Table).OpenRecordset(dbOpenTable)
.Index = Index
.Seek "=", Key1, Key2, Key3, Key4, Key5
If Not .NoMatch Then DSeek = .Fields(Field)
End With

End Function

Open in new window

Now we have som ething! DSeek() is over six tim es faster

than DLookup(), when used in a sim ilar context: a single
Úeld lookup on a table, using a sim ple criter ia on an
indexed Úeld, for exam ple:

DLookup("Com panyNam e", "Custom ers", "Com panyID

= 'OCEAN'")
DSeek("Com panyNam e", "Custom ers", "Pr im aryKey",
"OCEAN")

At this point it's good to read the help Úle on the Seek
m ethod.

Any index on a table can be used with Seek, and if the

index has m ultiple Úelds, one value m ust be provided for
each (at least when using the '=' oper ator). The table m ust
be opened in "table m ode", which is the default for the
"open r ecordset" m ethod of a "table deÚnition".

The Seek m ethod is only available in the "Microsoft Jet

wor kspace", which m ight include som e ODBC and ISAM
databases (but not ODBCDirect). I have not tested
external data sources in view of fast table lookups. If you
choose to do so, it's probably easier to use the ADO
library, which expose m ore properties and m ethods of
external database dr ivers.

A peek into ADO

ODBC drivers m ight expose Indexes and im plem ent the

Seek m ethod, and ADO will inform of their availability
through the supports pr operty. If
Suppor ts(adSeek+adIndex) returns tr ue for a recordset
opened as "table direct" with a ser ver -side "key set"
cursor, you can cr eate a function sim ilar to DSeek above.
[step=""]
Function DSeekADO( _
Field As String, Table As String, Index As String, _
ParamArray Keys() _
) As Variant

With New ADODB.Recordset

.CursorLocation = adUseServer
.Open Table, CurrentProject.Connection, _
CursorType:=adOpenKeyset, _
LockType:=adLockReadOnly, _
Options:=adCmdTableDirect
.Index = Index
.Seek Keys(), adSeekFirstEQ
If Not .EOF Then DSeekADO = .Fields(Field)
End With

End Function

6 de 9 Open in new window 05/07/2016 01:09 p.m.

Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

End Function

Open in new window

[/step]The function doesn't in fact call any exter nal driver ;

it sim ply uses the cur rent project's connection, m eaning
the curr ent Jet database. It does however dem onstrates
the ADO syntax of the Seek m ethod, and provides a
plausible candidate for benchm ar king against the
previous functions.

It turns out that the over head is m uch higher, sim ilar to
the problem when using CurrentDb, so that this attem pt
is only m ar ginally faster than DLookup (about 20%) when
tested on a Jet database. Jum ping ahead to the next
section and function (6), it is possible to create dedicated
lookup functions with ADO, with good results, but the
perform ance will not m atch that of DAO. Only one
exam ple is included in the attached benchm arking dem o
Úle, roughly Úve tim es slower than the equivalent DAO
function.

To put it sim ply, if you have a Jet database, use DAO. If

you have an external database, ADO seem s to deal better
with ODBC driver s, but bear in m ind that a better solution
to our goal -- a fast lookup function, is likely to m ake use
of stored procedures and sim ilar techniques rather than
r aw index m anipulation.

This opens an entirely new topic, but we have already

wander ed far enough fr om the intended scope of this
article. I will dr op ADO and dicussing external databases
at this point, and return to fast lookup functions in a Jet
database.

Dedicat ed Fast Look u p Fu n ct ion s

The best perform ance is achieved by im posing the m ost

r estrictions. We are now trying to obtain the value of a
single Úeld in a recor d, by m atching a value to an indexed
Úeld of the table, and we need to r em ove any rem aining
over head.

Our ver y Úrst attem pt used "CurrentDb.OpenRecordset".

Using "Cur rentDb" was a serious drag on perform ance,
but so is "OpenRecor dset", which needs to be elim inated
as well.
6
Function CompanyName(Key)

Static srec As DAO.Recordset

If srec Is Nothing Then

Set srec = DBEngine(0)(0)("Customers").OpenRecordset
srec.Index = "PrimaryKey"
End If

With srec
.Seek "=", Key
If Not .NoMatch Then CompanyName = !CompanyName
7 de 9 End With 05/07/2016 01:09 p.m.
Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

With srec
.Seek "=", Key
If Not .NoMatch Then CompanyName = !CompanyName
End With

End Function

Open in new window

The nam e of the function, Com panyNam e, is that of the

Úeld, !Com panyNam e -- notice the di˜ er ence. I often
r ecycle Úeld nam es as fast lookup function nam es.

The Úrst tim e the function is called, the static recor dset is
set to a "table-type" r ecordset of the table Com panies (the
"open r ecordset" m ethod defaults to "open table" instead
of "open dynaset"). This object will rem ain open between
function calls -- that's the m eaning of "static". The second
tim e, the function jum ps over the initialization.

For every call, the passed "Key" is sought in the cur rent
index. It doesn't m atter if the key Úeld is num er ic or text,
but the "Key" variable should be of the sam e type as the
key Úeld. See one typical trap and it's solution as Warning
just below the code snippet 7 below.

How does the function perform com pared to DLookup?

It's up to 100× faster ! We actually need to benchm ark it
against a Joined quer y.

Consider these three queries:

1. Jet perform s the JOIN:

SELECT O.OrderID, O.Custom er ID, C.Com panyNam e

FROM Or der s O LEFT JOIN Custom ers C ON
O.Custom erID = C.Custom erID
ORDER BY O.Or der ID;

2. Single-table query, calling Com panyNam e() externally:

SELECT OrderID, Custom erID

FROM Or der s;
(The lookup function is called from the loop, not the
query.)

3. Em bedded call to the function Com panyNam e()

SELECT OrderID, Custom erID,

Com panyNam e(Custom erID) As Com panyNam e
FROM Or der s;

Opening the quer ies 2. and 3., and br owsing their

r ecords, is of course slightly faster than doing the sam e
with 1., provided the function isn't called. So the test
should loop through all orders, sever al tim es, and actually
obtain the custom er 's com pany nam e for each order. In
cases 1. and 3., this m eans reading the nam e into a
dum m y variable -- this forces Jet to evaluate either the
join or the calculated Úeld. In case 2. this m eans calling
8 de 9 05/07/2016 01:09 p.m.
Access Techniques: Fast Table Lookup Functions https://www.experts-exchange.com/articles/1921/A...

cases 1. and 3., this m eans reading the nam e into a

dum m y variable -- this forces Jet to evaluate either the
join or the calculated Úeld. In case 2. this m eans calling
the function "externally" from code.

The raw m easure is the num ber of turns in the loop

during one second, which includes wr iting the com pany
nam e into a variable. The order of m agnitude on m y old
laptop is 1e5, m eaning that execution tim es are
expr essed in µs.

Their relative r aw perfor m ance is roughly equal to their

num bering in the list: the second is two tim es slower than
the Úrst, and the thir d three tim es.

This is im pressive! Of course Jet perform s better when

joining two tables on key Úelds than anything we can do
from the outside. The overhead is in the function call
itself. But the point is that this solution's scores in the
sam e order of m agnitude as plain SQL joins.

If we subtr act the tim ing of an em pty loop (writing a Úeld

from the Orders table, and not the linked Com pany
Nam e), and com pare the rem aining tim e (a plausible
m easur e of the real tim e spent), the com par ison rem ains
honour able: around 7 tim es between cases 1. and 3.

9 de 9 05/07/2016 01:09 p.m.