The 2017 4th International Conference on Systems and Informatics (ICSAI 2017)

An Empirical Study of Three PHP Frameworks

Xiaosong Li, Sai Karnan
Jahanzaib Ali Chishti
Computer Science Practice Pathway
Department of Electrical and Computer Engineering
Unitec Institute of Technology
University of Auckland
Auckland, New Zealand
Auckland, New Zealand

Abstract— In recent years, there are many PHP frameworks for most suitable framework with the best support features to use in
developers to choose. The developers should know how to choose their project. For that reason, good understanding of different
the most suitable framework with the best support features to use frameworks become an essential requirement for web
in their project. For that reason, good understanding of different developers nowadays.
frameworks becomes an essential requirement for web developers
nowadays. This study conducted a series of experiments to This issue usually ends up with questions like: What is the
compare the performance and reusability of three selected best framework to use? Or, what are pros and cons of different
frameworks by implementing the same token web application PHP frameworks? How to know which framework is the most
using three PHP frameworks: CakePHP, Laravel and CodeIgniter suitable for a project? And many more.
respectively. In terms of performance, CodeIgniter performed the
best for the three tasks. Laravel performed the best for one of the For that purpose, this study conducted a series of evaluations
tasks and performed the worst for one of the other tasks. In and experiments to compare the performance, efficiency and
general, CodeIgniter has the best performance and Cake PHP has reusability on three selected frameworks that were used in
the worst performance. In terms of reusability, CakePHP has the workplaces popularly in 2013 to help web developers
highest score. This suggested that the performance and the understand PHP frameworks and making decisions in selecting
reusability are not in proportional relationship. There could be a PHP frameworks for their projects. [2]. Most comments
number of reasons for this. Regular expressions were used to regarding PHP frameworks are based on the subjective
detect reusability data from the code. These regular expressions observations and qualitative analysis. Recently, there were a few
cannot deal with repeat variable occurrences which may result in studies on PHP frameworks evaluations and comparisons. The
inaccuracy of the reusability score. Further research is required. study of [3] explored the relationship between a framework and
In the future, more frameworks should be included to get the the effort it takes to develop a web application. The study of [4]
general guidelines for PHP framework evaluation. More factors compared and analyzed the impact of data abstraction layer
should be considered in the PHP framework measurements. (ORM) on the performance of two PHP frameworks CakePHP
and CodeIgniter.
Keywords-PHP frameworks; experiment; measurement;
evaluation; comparison The intention of this research was to work out a complete set
of measurements for the evaluation of six popular frameworks,
I. INTRODUCTION which can be further refined into a PHP framework evaluation
PHP is considered as one of the most popular scripting formal procedure. Ultimately, a guideline for developers in
languages to be used in developing web applications due to the choosing and applying frameworks in their workplace or
fact that it is dynamic, provides a large amount of flexibility, it’s personal projects will be provided. This paper presents the first
easy to use and is easy to learn [1]. However, sometimes coding phase of the research, where three frameworks were considered,
in PHP become a repetitive task. Moreover, the easiness to apply and the measurements were focusing on the performance and
PHP into projects also leads inexperience developers to reusability. The results should be useful for PHP developers and
unorganized code process that they do not even notice. could be improved in the next phase of the research.
PHP frameworks come into the picture to help developing By implementing the same token web application using three
web application smoother and faster by providing a template selected PHP frameworks: CakePHP, Laravel and CodeIgniter
basic structure, along with a completed set of built-in APIs, respectively, the research provided understanding on how each
libraries and numerous of extensions (which are developed by framework was used in process and the technical details to
framework creators or the community). In other words, evaluate the performance of each framework using time
frameworks can speed up developing process, cut off developing measurements for different tasks. The results were analyzed
time, help developers be more productive by reducing repetitive qualitatively.
code in the project. The final application created by frameworks In the rest of this paper, the related research is discussed first,
will be more stable and secure. which is followed by the experiment design, data collection and
In recent years, there are many PHP frameworks for analysis. Finally, the summary and future work are given.
developers to choose from such as Laravel, CakePHP and
CodeIgniter etc. In fact, in real world working environment, II. RELATED RESEARCH
developers will not write every single line of code while building Five general criteria that developers use to select a
applications, but instead they use frameworks to optimize their framework were indicated by [3]. They are architecture,
tasks. Therefore, developers should know how to choose the documentation, community support, flexibility or whether they

978-1-5386-1107-4/17/$31.00
Authorized ©2017 IEEE
licensed use limited to: UNIVERSIDADE 1636on July 24,2024 at 18:52:46 UTC from IEEE Xplore. Restrictions apply.
DE SAO PAULO. Downloaded
possess a list of features. While these criteria are essential, they (a) Number of include and Number of require (ܰ‫)݌݄݌ܴܫ‬
are not task specific and lack of implementation details. They
don’t provide deeper help on making decision for a concrete web ܰ‫ ݌݄݌ܴܫ‬ൌ σܰ‫ ݌݄݌ܫ‬൅  σܴܰ‫݌݄݌‬ (1)
application. In [3], a method using source lines of code as a
comparative metric to evaluate PHP MVC frameworks and where ܰ‫ ݌݄݌ܫ‬ൌnumber of include files and ܴܰ‫ ݌݄݌‬ൌ number
determine the differences in effort required to implement the of require files.
fundamental components of a web application was developed.
(b) Number of object inheritance elements (ܱܰ‫)݌݄݌ܫ‬
According to [4], PHP frameworks can considerably
improve the performance of an application. These frameworks ܱܰ‫ ݌݄݌ܫ‬ൌ σܰܶ‫ ݌݄݌‬൅  σܰ‫ ݌݄݌ܧ‬൅ σܰ‫݌݄݌ܥܥ‬ (2)
usually are based on Model, View, and Controller design
pattern. These frameworks provide, different common where ܰܶ‫ ݌݄݌‬ൌnumber of include files; ܰ‫ ݌݄݌ܧ‬ൌ number of
functionalities and classes in the form of helpers, components, extends and ܰ‫ ݌݄݌ܥܥ‬ൌ number of concrete class.
and plug-in to reduce the development time. Due to these
features such as robustness, scalability, maintainability and (c) Class Interface Size (‫)݌݄݌ܵܫܥ‬
performance, these frameworks are mostly used for web
development in PHP, with performance being the most ‫ ݌݄݌ܵܫܥ‬ൌ σ‫݈ܾܿ݅ݑ݌‬ሺܱܰ‫ܯ‬ሻ ൅  σ‫݈ܾܿ݅ݑ݌‬ሺܸ‫ܴܵܣ‬ሻ (3)
important factor. [4] reported the research on comparing and
analyzing the impact of data abstraction layer (ORM) on the where ‫݈ܾܿ݅ݑ݌‬ሺܱܰ‫ܯ‬ሻ ൌ number of public methods and
performance of CakePHP and CodeIgniter. Load testing and ‫݈ܾܿ݅ݑ݌‬ሺܸ‫ܴܵܣ‬ሻ  ൌ number of public variables.
stress testing were used to measure the performance. The results
of [4] showed that there is not much of a difference, as per as the (d) The values of the above defined metrics will become base
performance of CakePHP and CodeIgniter was concerned, with attributes to evaluate reusability (ܴ‫ )݌݄݌ܷܧ‬of PHP codes.
the respect to response time on a live server. CodeIgniter had
better performance with respect to throughput on live server. ܴ‫ ݌݄݌ܷܧ‬ൌ σܰ‫ ݌݄݌ܴܫ‬൅  σܱܰ‫ ݌݄݌ܫ‬൅ ‫݌݄݌ܵܫܥ‬ (4)
[5] indicated that although PHP dynamic features can give
The above formula had been used to calculate the reusability
the developers great flexibility, they could have a negative
measurements of the PHP framework applications in this study.
impact on performance.
[8, 9] indicated that web application frameworks usually
According to [6], database connection is very important for
implement the Model View Controller (MVC) design pattern.
a PHP framework. What happens when multiple parts of your
The MVC pattern is a proven and effective way for the
application need to interact with the database? Surely that
generation of organized modular applications. The MVC
database-related code shouldn’t need to be duplicated in each
pattern breaks an application into three modules: model, view
and every PHP script. The prudent thing to do would be to
and controller. The model module contains the underlying
refactor all database code into a shared PHP file. A developer
classes whose instances are to be used for manipulating the
should really need to worry about opening and closing database
database, templating frameworks and session management, and
connections. The database-related tasks should be included in
they often promote code reuse. All the three PHP frameworks
the PHP framework evaluation experiments.
selected for this study implement MVC design pattern.
In software engineering, software metrics are the only tools
to control the quality of software [7]. Most of the metrics are III. EXPERIMENT DESIGN
developed covering generally programming languages such as Empirical study is based on observed and measured
C, C++, and Java etc. Some metrics may not be suitable for some phenomena and derives knowledge from actual experience
programming or scripting languages. There is lack of metric in rather than from theory or belief [10]. A PHP system was
the literature, which measures the quality of PHP language [7]. developed to conduct the experiments. It consists of a control
In this respect, specific metrics should be developed, and we panel and three PHP applications. The PHP applications are the
develop a metric for PHP, which is capable to calculate re-implementation of the same token application by using
reusability quality of PHP code. A reusability metric to measure CakePHP, Laravel and CodeIgniter frameworks respectively.
quality of PHP script language (REUphp) was proposed by [7]. The control panel allows the users to conduct the experiments
Since PHP is an object oriented language, the present metric is and to collect data. The same set of measurement points were
capable to evaluate any object-oriented. integrated with each of the applications. The databases for each
Software reuse is the process of implementing or updating application were identical as well. Fig. 1 shows an overview of
software systems using existing software components. the experiment system architecture. The system was
Reusability of Object Oriented Programming is the use of implemented by PHP 5.6 on windows operating system and IIS,
previous classes or functions or methods in the present class but and Database 1, Database 2 and Database 3 were implemented
no problem of previous classes. The goal of Reusability quality by MySQL server. The management system included an UI
of PHP code (REUphp) is to evaluate the current state of the art allowing access to the applications implemented by CakePHP,
on the reuse metrics area with special emphasis on code-based Laravel and CodeIgniter respectively.
metrics. It includes the following factors:

1637
Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on July 24,2024 at 18:52:46 UTC from IEEE Xplore. Restrictions apply.
 Another piece of software called RIPS (http://rips-
scanner.sourceforge.net/) was used to extract and calculate the
reusability measurements. After the URL of the PHP application
is provided, the measurements defined in formula (1), (2), (3)
and (4) of [7] will be produced. Regular expressions defined in
Table I were implemented to help extract the measurements.

TABLE I. REGULAR EXPERESSION USED IN THE EXPRIMENT

Purposes of the
Regular Expressions
REGEX
include or require ^((include)|(require))

extends extends

trait ^trait
functions without
visibility modifiers ^\s*function[\s\n]+(\S+)[\s\n]*\(
functions with public
modifiers ^\s*public\s*function[\s\n]+(\S+)[\s\n]*\(
Figure 1. An overview of the experiment system architecture. \$([a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-
ALL the variables \xff]*)\s*=
variables with private (protected\s*\$([a-zA-Z_\x7f-\xff][a-zA-Z0-
The token application integrated minimal necessary features or protected visibility 9_\x7f-\xff]*)\s*=)|(private\s*\$([a-zA-Z_\x7f-
of a web application. These features including: uploading of modifiers \xff][a-zA-Z0-9_\x7f-\xff]*)\s*=)
files, connecting with database, populating data from database, concrete classes ^class\s*[\s\S]+
data validation, inserting data into a database, using master page
as a template etc.
Table I shows the regular expressions used to measure the
The experiments could help us to investigate benchmark code reusability in the various cases of a PHP application as
tasks for next step of research. The following are the tasks used defined in [7], where ^ denotes the starting position of the line,
for performance measurements in this study: hence can be used with 'include' and 'require' etc., but not with
'extends'. There are limitations for the regular expressions
• Read/write text files.
defined for variables; they count all the instances of the variable
• Upload and save images to the web server. occurrences, e.g. $var1=x; $var2=m; $var1=p; will be counted
• Retrieve large amount of data from the database and as 3 not 2, as the regular expressions cannot deal with repeat
display them in a table. variable occurrences (i.e. $var1 was counted twice). The
• Database CRUD operations (connecting to/insert measurements obtained based on these regular expressions
into/delete from/update the database). contributed to the final reusability measurement for a PHP
application by using the formula (1), (2), (3) and (4) defined in
Fig. 2 shows the web page from the token application used the [7].
for large amount data (complex data) processing measurements.
IV. DATA COLLECTION AND ANALYSIS
In the experiments, identical data were used for all the three
PHP applications. The resulting data were collected from the
web pages or the resulting excel files. The results are presented
in the tables in this section.
A. Reusability Results
Table II shows the final experiment results of reusability for
three PHP application using three PHP frameworks CakePHP,
Laravel and CodeIgniter respectively. REUphp indicates the
reusability of a PHP application. NIRphp represents the number
of include and the number of require used in the PHP code.
NOIphp represents the number of object inheritance elements
(traits, extends and concrete classes) in the PHP code. CISphp
represents the Class Interface Size (public methods and
variables) in the PHP code.

Figure 2. A web page from the token application.

1638
Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on July 24,2024 at 18:52:46 UTC from IEEE Xplore. Restrictions apply.
 TABLE II. REUSABILITY RESULTS performed the fastest of the three in this case. This could be due
Framework Used NIRphp NOIphp CISphp REUphp to that CodeIgniter is the simplest of the three PHP frameworks
CakePHP 97 5896 78331 84324 considered in this study, therefore, the framework data
transferring overhead for this framework is small, thus making
Laravel 116 7388 51874 59378 CodeIgniter more efficient.
CodeIgniter 1 246 5466 5713 C. CRUD Operations
In this task, three database operations (create a record, update
a record and delete a record) were tested for each of the PHP
Table II shows that the PHP application implemented in applications. The same data structure and testing data were used
CakePHP framework has the highest reusability score 84324 and for all the applications. Table V shows the results for CRUD
the PHP application implemented in CodeIgniter framework has Operations.
the lowest reusability score 5713.
TABLE V. CRUD OPERATIONS
TABLE III. REUSABILITY MEASUREMENTS
CakePHP Time Laravel Time CodeIgniter Time
Operations
Measurements CakePHP Laravel CodeIgniter (milliseconds) (milliseconds) (milliseconds)
files scanned 3763 4700 212 Create 180.85 150.44 98.22
include or require 97 116 1 Update 40.62 27.27 12.27
extends 2480 3266 110 Delete 28.46 14.44 6.74
trait 115 110 0

functions without 989 582 166 CodeIgniter application performed the best for the three
visibility modifiers database operations. CakePHP application performed the worst.
functions with public 20729 19016 997
This suggested that the database component of CakePHP
visibility modifiers
public functions 21718 19598 1163 framework has lower performance than the database component
of Laravel framework.
ALL of the variables 57899 33926 4626
D. Upload Images
variables with private or 1286 1650 323
protected visibility A file upload box was used to test the performance of the
modifiers frameworks when an image is uploaded to the web server. Upon
56613 32276 4303 uploading of the image, a report was generated on the server
public variables
(and stored). This report was also displayed on the browser when
3301 4012 136
concrete classes the image uploading is finished. A set of 10 image files were
selected for testing. The uploaded files were saved to the web
server. Table VI shows the results for uploading and saving
Table III shows the detail measurements for the PHP images.
applications implemented in different frameworks. For
CodeIgniter application, the number of files scanned were the TABLE VI. UPLOAD IMAGES
smallest (212), as it is a simple PHP framework. However, it has
File Size CakePHP Laravel CodeIgniter
used extremely small number of include or require and trait, File
(KB) Time Time Time
and these reduced the code reusability in CodeIgniter framework Name
(sec) (sec) (sec)
and their applications. The CakePHP application has fewer files 1.jpg 89.83 0.634 0.318 0.605
scanned (3763) than the Laravel application (4700), however,
the CakePHP application used more functions, private or 2.jpg 197.202 1.187 1.541 1.32
protected modifiers and public variables, these made CakePHP 3.jpg 309.747 1.683 2.22 1.954
framework and its applications have better code reusability than
Laravel framework and its applications. 4.jpg 416.139 2.186 2.664 2.672

5.jpg 623.068 5.442 4.702 4.679

B. Complex Data
6.jpg 1119.395 9.306 8.801 8.321
TABLE IV. COMPLEX DATA
7.jpg 1398.764 10.654 9.991 10.319
CakePHP Time Laravel Time CodeIgniter Time
Operations
(milliseconds) (milliseconds) (milliseconds) 8.jpg 1636.173 13.587 12.574 11.676
Load 40.2 65.12 18.91
9.jpg 1810.801 15.3 13.894 13.192
Display 1.7 7.32 0.022
10.jpg 2046.306 14.58 15.462 14.379

Average 7.4559 7.2167 6.9117

Table IV shows the time for loading data from the database
and displaying the data on an HTML table. Laravel application
performed the worst in this case. CodeIgniter application

1639
Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on July 24,2024 at 18:52:46 UTC from IEEE Xplore. Restrictions apply.
 On average, CodeIgniter application performed the best for relationship. There could be a number of reasons for this. The
uploading and saving image files. CakePHP application regular expressions cannot deal with repeat variable
performed the worst. So, the file uploading and saving occurrences; this may result in inaccuracy of the reusability
component of CakePHP framework has the lowest performance. score. Further research is required to produce more accurate
regular expressions.
E. Read/Write Files
Eight text files were used for testing the reading/writing files. In the future, more frameworks should be included to get the
Each of the file was copied (read and written), line by line to general guidelines for PHP framework evaluation. The
another file. The copying of the file was timed. A report was following factors should be considered in the PHP framework
produced at the end containing the time taken for copying each measurements.
of the eight files. The results are as shown in Table VII. • Investigate the benchmark tools for measurements
TABLE VII. FILE READ/WRITE
• Learning curve.
• Neat & clean intuitive API.
File Size CakePHP Laravel CodeIgniter
File
(MB) Time Time Time • Scalability.
Name
(millisec) (millisec) (millisec) • Security.
1.txt 2.506204 65.94 52.89 63.93
• Solid code written on proven design patterns.
2.txt 5.087024 110.7 103.04 115.18 • Quality of documentation.
3.txt 10.173506 209.5 224.13 207.34 • Security vulnerability
4.txt 20.347014 582.5 434.83 415.54 REFERENCES
5.txt 40.69403 1114.37 845.98 865.99 [1] M. Hills, P. Klint, and J. Vinju, "An empirical study of PHP feature usage:
a static analysis perspective," Proceedings of the 2013 International
6.txt 81.388062 1696.98 1706.8 1653.79 Symposium on Software Testing and Analysis. ACM, 2013.
162.77613 3402.53 3338.44 3292.38 [2] B. Skvorc, “Best PHP Frameworks for 2014,”, December 2013, Sitepoint,
7.txt https://www.sitepoint.com/best-php-frameworks-2014/, last accessed
8.txt 325.55225 7034.43 6680.45 6850.06 24/10/2016.
[3] D. Clare, “A comparative evaluation to measure effort between PHP-
Average 1777.1188 1673.32 1683.02625 based frameworks,” Project Report, 2005.
[4] A. Fayyaz and M. Munir, "Performance Evaluation of PHP Frameworks
(CakePHP and CodeIgniter) in relation to the Object-Relational Mapping,
On average, Laravel application performed the best for with respect to Load Testing," 2014.
reading and writing text files. CakePHP application performed [5] C. Mulder, “Reducing Dynamic Feature Usage in PHP Code,” Master’s
the worst. So, the file processing component of Laravel thesis, University of Amsterdam, 2013.
framework has the best performance. [6] Laravel Book, “Laravel Introduction,” http://laravelbook.com/laravel-
introduction/, last accessed 26/10/2016
V. SUMMARY AND FUTURE WORK [7] C. T. Mon and K. M. Myo, "Framework for Evaluating Reusability of
PHP," International Conference on Advances in Engineering and
In terms of performance, CodeIgniter performed the best for Technology (ICAET'2014), pp25-59. March 29-30, 2014 Singapore.
Complex Data, CRUD Operations and Upload Images tasks. [8] I. Sarker and K. Apu, "Mvc architecture driven design and
Laravel performed the best for Read/Write Files and performed implementation of java framework for developing desktop application,"
the worst for Complex Data. This could be due to that International Journal of Hybrid Information Technology 7.5 (2014): 317-
CodeIgniter is the simplest PHP framework with the smallest 322.
files numbers among the three selected frameworks in this study; [9] A. Paikens and G. Arnicans, "Use of design patterns in PHP-based web
application frameworks," Scientific Papers University of Latvia,
therefore, the framework data transferring overhead is small, Computer Science and Information Technologies 733 (2008): 53-71.
thus more efficient. In general, CodeIgniter has the best [10] Penn State University Libraries, “Empirical Research in Education and
performance and Cake PHP has the worst performance. In terms the Behavioral/Social Sciences,” http://guides.libraries.psu.edu/emp , last
of reusability, CakePHP has the highest score. This suggested accessed 30/10/2017.
that the performance and the reusability are not in proportional

1640
Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on July 24,2024 at 18:52:46 UTC from IEEE Xplore. Restrictions apply.