2010 International Conference on Intelligent Computing and Cognitive Informatics

Use of SQLite on Embedded System

Dai Chen, Xudong Han, Wei Wang

School of Computer Science
China University of Mining And Technology
Xuzhou, China
{chend, weiw}@cumt.edu.cn, [email protected]

Abstract:-This design is for transplanting and using SQLlite the embedded operating system μCLinux, and has played a
database management system on μC/OS-II, the embedded real- significant role in data management in embedded devices.
time operating system. The design has used μC/OS-II, the open- μC/OS-II is a short but powerful real-time embedded operating
source operating system, μC/FS, a file system transplanted on system, and has a wide range of applications in embedded area,
μC/OS-II, and the sqlite database. By studying their design but it is just a real-time kernel and is difficult to meet the
principles and methods, clarifying the overall structure and the complex data processing requirement [2]. This paper transplants
links between the various parts, I try to design a specific SQLite to μC/OS-II and tests the result of the transplantation
application in series with them. The design is divided into four with an application.
modules: μC/OS-II operating system module, μC/FS file system
module, sqlite database module and the final application module. II. μC/OS-II RELATED FILES MODIFICATION
The final system can be achieved in operations of addition,
deletion, search and update of data and so on in the multi-task μC/OS-II is an open source operating system with features
scheduling controlled by μC/OS-II. The operation of various of simplicity, refinement, readability, cuttability, portability,
commonly used functions can also be included in the final system. ability to be solidified ， preemption, etc. It completes the
following basic function of tasks scheduling, tasks
Keywords-μC/OS-II; μC/FS; SQLite;transplant; embedded management, time management, interrupt management,
database memory management and communication and synchronization
between tasks.
I. INTRODUCTION The architecture of μC /OS-II is shown in Fig. 1.
With the rapid development of electronic technology,
particularly, the microcomputer based on the large-scale
integrated circuit arising a qualitative leap in modern scientific
studies, the embedded microcontroller technology has brought
a revolution to the modern industrial control area. The most
obvious advantage in the embedded micro-controller system is
that you can embed it in any micro or small scale instrument or
equipment. Therefore, the application of embedded systems
will become increasingly important. Driven by the technology
of microelectronics and storage, the memory in the embedded
system and permanent storage media capacity are becoming
larger and larger, and the embedded system applications are
expanding. That means that the amount of data to be dealt with
in the embedded system will continue to increase, so large
amount of data processing becomes very necessary. We have to
bring the complex database processing technology originally Figure 1. The Architecture of μC/OS-II
used in the enterprise-class to the embedded system, that is
database technology in the embedded system [1]. Some files of them need to be modified or rewritten to
Although derived from large database management adjust to the new environment for this design, or to improve the
technology, the embedded database technology has its own efficiency.
features such as small storage space, high reliability, easy
manageability, good security, cuttability and portability and so A. Assembling files modification
on which is decided by the characteristics of the specific What we need to do is, just like all other transplants, to
embedded system. So we must change and improve some parts modify the μC/OS-II processor-related code, which is mainly
of the system based on the original large database system. At included in three files: OS_CPU_A.ASM, OS_CPU_C.C,
present, there have been some mature databases such as SQLite OS_CPU.H. We choose to use the language of C and the
and BerkeleyDB which are suitable for embedded compiler of Microsoft Visual C++ 6.0. We have to rewrite the
environment. SQLite database management system is written assembling file OS_CPU_A.ASM and put it into a new C file
in C, and its source is open. With the advantages of little because the compiler, VC 6.0, cannot separately compile the
resources, usually a few hundreds of Kilo Bytes, fast speed, assembly code, other for embedded into C files. Here, this
high efficiency, cuttability, it has been ported successfully to design refers to the C file which is already rewritten by net

978-0-7695-4014-6/10 $26.00 © 2010 IEEE 210

DOI 10.1109/ICICCI.2010.79
friends ， adding the rewritten contents of the original The “ram” (or array) will be recycled when the application
OS_CPU_A.ASM to OS_CPU_C.C for compiling. closes, and the files stored in the “ram” will be destroyed. It
means that the original data stored in that array last time is not
B. Hardware timer simulation available when this application restarts. Transferring data to the
hard disk before the end of the process seems a good idea, and
μC/OS-II achieves the system clock with the help of the load the contents from the hard disk to the “ram” every time the
hardware timer which generates a periodic interrupt every application is started. This requires a file in the hard disk as an
period of millisecond (ms). The smallest unit of time is the image of “ram”, named “ram.img”. In fact, nobody knows
interval between interrupts, and is called the clock beat. But when this application ends. Therefore, maintaining the same
how can we get “clock beat” from VC? This problem must be contents of ram.img and ram is necessary to ensure the data is
the first consideration. In windows protected mode, unlike in not lost. That means every time the data in ram changes,
DOS directly through a function call to modify the interrupt, it ram.img should also be changed accordingly.
means to involve the driver to modify the interrupt, and
difficulty of porting and complexity will be highly increased,
but only the real hardware clock “tick” can ensure the real-time IV. SQLITE TRANSPLANT
of μC/OS-II. Another alternative solution is to use the software SQLite is an excellent embedded database engine. It
timer to generate a simulated clock “tick”. This design chooses supports most of SQL92 standard syntax, and its architecture
the latter because its purpose is for study not for real-time primarily consists of 8 modules composed of three sub-
product. There are many types of software timers in windows, systems, as is shown in Fig. 3.
for example, calling the windows API function timeSetEvent
can define periodic tasks in the callback function so as to
Interface B-tree
compete the necessary processing of the event. Calling this
function will add a thread, and the callback function will be
called when the time comes. It is very similar to the external Tokenizer Pager Backend
interrupt for the main thread. That is precisely what we need [3].
Core
OS
III. μC/FS ARCHITECTURE Parser Interface

Conducting complex data access in μC/OS-II requires the Compiler

support of the file system. Here, we choose the μC/FS. This Code
Generator
embedded file system is designed by U.S. micrium Inc.,
particularly for embedded applications and is compatible with Database
Virtual
all types of hardware. It is designed to support FAT12, FAT16 Machine
and FAT32, and applied to all storage media by providing users
with basic hardware access functions. It is optimized in speed, Figure 3. The Architecture of SQLite
versatility and memory packages and is a high performance
library. Its logical structure is shown in Fig. 2. The module of API interface provides functions for
external. The Word Breaker (Tokenizer) and Analyzer (Parser)
in the compiler check the syntax of the SQL sentence, and put
the sentence into the hierarchical data structures —syntax tree,
which is facilitate for following parts. The Code Generator
produces a type of assembly code for SQLite according to the
syntax tree, and the Virtual Machine implements the code.
Virtual Machine is mainly an abstract computing engine
designed for the operation of the database. The Back Part
includes the B-tree, page cache, and the operating system
interfaces. B-tree is mainly to support the index, maintain the
complex relationship between the various pages, and find the
required data easily and quickly. The main role of the page
cache is to transfer pages between B-tree and the hard disk
through the operating system interface.
Figure 2. The structure of μC/FS
All I/O operations in SQLite are eventually converted into
operating system calls. SQLite source code has provided
Using specific hardware needs the driver of the hardware. operating system interface functions for Windows, WinCE,
The driver is supposed to include the underlying I/O functions Unix, OS2, but not for μC/OS-II. Corresponding functions
to access this hardware and the global table to store the pointers need to be added, which is difficult and complex. Fortunately,
to those I/O functions [4]. In this design, an array in memory is there are little operating system restrictions on SQLite. With
designated as a storage device, called “ram” (however, not real the help of SQLite source code for Windows and μC / FS,
hardware device). So the driver code is mainly the operation of interface functions for μC/OS-II are implemented.
the array, including functions of query of the device status, In the process of rewriting the code of interface functions,
reading and writing, and controlling the device. SQLite source code for Windows which are mainly in

211
os_win.c, is greatly referred. However, μC/OS-II is quite compiled in large mode. So we choose Microsoft Visual C++
incomplete compared with Windows, many data structures and 6.0.
functions in windows cannot find their counterparts in μC/OS- 3) Memory Allocation
II. That brings a lot of trouble to rewrite them. The author “ram” is not real, but an array existing in the memory.
rewrites most of functions needed according to his own
Memory is allocated through the array rather than the fixed
understanding, but with simplification, and may make the
database much less colorful. memory block in μC/OS-II. So drives for the storage devices
must be rewritten when this system is transplanted to a
particular processor, and the start address of the ram array
V. APPLICATION DESIGN
should be set to the non-volatile storage start address, so as to
The ultimate goal of the application is to verify the usability save the data in SQLite database.
of SQLite database on μC/OS-II. This application tests the 4) Minimum System
common functions, including the commands of create The size of memory required for this system = memory
(construction of a table), insert, delete, update and so on, but
not all the functions in the database are tested. The application needed in the testing application + memory needed in kernel
also checks the availability of μC / FS. data area + (size of TCB, stack of tasks and maximum
This design achieves the above functions with the following interrupt nesting stack ) * number of tasks [5];
methods: firstly the user’s command is received, secondly the memory needed in the testing application + memory needed
type of the command is determined according to the first string, in kernel data area <1.5M (shown in the task management of
thirdly an appropriate function according to this type of windows )
command is called to process this command. If the command is size of TCB + stack of tasks + maximum interrupt nesting
about file operations, the corresponding function in μC / FS stack
will be called, and if a database operation, a function in SQLite < ( 0x3c+2048*4+0 ) byte < 8.5KB;
will be called. And if an error happens, the appropriate The size of memory required for this system < 1.5MB +
information will be output.
8.5KB * 64 < 2MB;
In μC/OS-II operating system, the global variable
OS_LOWEST_PRIO is defined as 63, which means the system So the space of the volatile part of the storage device should
can manage 64 tasks. μC/OS-II supports multi-task be more than 2MB, and the nonvolatile part should be
management, and its kernel is preemptive, so the ready task compatible with the amount of data stored in the database.
with the highest priority will always gain the opportunity to
run. The priority numbers 0 - 63 are considered as their ACKNOWLEDGMENT
separate identification, in which idle task takes priority of 63, In the process of working on designing and programming
with the lowest priority, and statistics task takes 62, the two this thesis, I have gained a lot of guidance of teachers in our
tasks are the system tasks. Tasks with the priority of 0-5 are school and much help from my friends. I award a lot to them,
reserved. Task 6 is set to start the timer by creating a new especially my supervisor Mr. Chen, who has given me very
thread to monitor the main thread, in which the Windows API important suggestions and guidance in critical time, and has
function timeSetEvent is called and a hardware timer is never failed to encourage me to study hard to conquer troubles
simulated. Task 7 is the user’s first available task, so 54 tasks and achieve my success, in this way I was able to continue my
can be used for users, their priorities are from 7 to 61.Users can work. Every time I got frustrated in my present job, he could
input “task *” to start this task, and input some commands next. always point out the problem and showed me the right
direction. Patiently to explain to me in detail, his words gave
VI. CONCLUSION AND DISCUSSION me great confidence and courage to finalize this design. At this
This paper successfully transplants the SQLite embedded point, I want to express my sincere gratitude to Mr. Chen and
database to embedded operating system μC/OS-II, and verifies other helpers!
the basic functions of the system through an application. The In addition, this design has got a lot help and support from
result reflects the platform independence and powerful students who are interested in related topic and some internet
database management features of SQLite, and makes it possible friends. Their active participation to this subject has brought
to transplant database and deal with complex data. However, limitless expectations and vitality to my job. Through the
this study still leaves some problems to discuss: exchange, I has got not only fresh ideas to expand the field of
1) Windows API vision but also friendship. Thanks to their participation and
Windows API function timeSetEvent is used to simulate the accompany, and much thanks and best wishes to all students
hardware timer in a new thread which monitors the main and teachers who have ever contributed to my work.
thread. This section of code should be rewritten when it is
transplanted to a specific processor. REFERENCES
2) Compiler [1] The current situation of embedded database.
http://www.pvontek.com/blog/arm/14.html
The space of variables is limited in some compilers such as
[2] Mingxing Wu,Tong Li.SQLite Porting on uC /OS-II. Science
Borland C 4.5 in large mode, while the SQLite database Technology and Engineering, 2009,4:1023-1027.
requires many variables. If Borland C 4.5 is used in huge [3] The Transplant of μC/OSII under the VC.
mode, it is not compatible with the OS_CPU_A.ASM, which is http://download.csdn.net/source/1293584

212
[4] Transplant of embedded file system uc / fs into the Nios II based on Nor [5] Embedded real time OperatingSystem μC/OS-II and Application.
Flash. http://www.edacn.net/html/92/82692-7979.html http://www.embed.com.cn/downcenter-/Article/Catalog10/2528.html

213