A Windows Based Photo Booth Application Written in C++

John Paul Swaynos

CEN4914, Senior Project CISE Department University of Florida

Advisor: Dr. Eric M. Schwartz, email: [email protected] Machine Intelligence Laboratory University of Florida, Gainesville, FL 32611

Date of Talk: 7th December 2011

Abstract There currently are limited options when selecting a Windows based application to emulate Photo-Booth behavior. On Apple OS X, there is a particular piece of software written by Apple called Photo Booth which captures the experience of a physical Photo Booth into code. On Windows, however, there is no go-to application to match this kind of behavior. There are a couple of pieces of software, namely Cameroid and Seenly, which communicate with a connected webcam using Adobe Flash and run in a web-browser, these products are very similar to Photo Booth for OS X, but they require internet access to work. There should be a product available to the Windows Operating System that emulates a Photo Booths behavior, not requiring web-access, and does so in an intuitive manner that anyone can understand. In response to this situation, I have developed a Windows application using Microsofts Visual Studio 2010 and the C++ programming language. It operates in two distinct modes, an Easy Mode which provides basic options, and a dedicated Photo and Printer Mode in which two instances of the application work together in a client/server pattern and relies on network communication. Having no prior experience in writing C++ code, I followed a waterfall development model. In which the application was broken down into sequential components that could only be completed one after the other, and couldnt be completed until I understood what was needed to get it to work. In the beginning I relied on rapid prototyping, in which I got the fundamental components functioning together, and this worked well on building my understanding of C++ and programming with the Windows Operating System. Yet by using rapid prototyping I locked myself into one method of creating graphical user interfaces (GUI) early on. Additionally, this decision caused me to adopt an outdated library to connect to the webcam hardware, although my code could be drastically modified to link to another, more recent, library. Introduction I will be getting married within the next year, and one thing my fiance and I really wanted to have at our wedding is a Photo Booth. Unfortunately photo booths are very pricy to rent, and doing so would provide additional issues regarding logistics. It was decided that the easiest way to have a photo booth at our wedding, but not spend a lot of money, is to use a computer, printer, and a webcam. Regrettably, there isnt a standalone application that allows you to select the number of shots you wish to take, and there certainly isnt an application that

makes the process easy. There are other available online applications, but they require webaccess. A luxury which we are not guaranteed at our venue. Therefore the best solution is a standalone Windows application. This application needs to have an intuitive and easy user interface, so that guests of any age can dive in and use the application without assistance, and it needs to take the shots and then automatically process the print. Although there is a lack of photo booth software for Windows, there are many applications that make use of webcams. Because of this demand, Microsoft provides multiple libraries that allow one to hook up a webcam to their application. Two libraries that I became familiar with are Video for Windows, and Direct Show. Direct Show is the successor to the Video for Windows library. Using the Video for Windows library an online tutorial shows how to create a simple program that connects to a webcam, and captures bitmaps using the Windows Graphics Device Interface, or GDI library. This tutorial was easy to follow, and formed a base for my first prototype. This code is the reason why I made many of the design decisions that plagued me later on. There is also another project, called Photo Boof, which overcomes many of the problems that led to this project, but it lacks the networking capabilities that my project provides. In solution to this proposed application I, in assistance of my advisor, drafted a proposal of components that this project needs to have. It needs to have the three modes that the program can run in, Easy Mode, Photo Mode, and Printer Mode. It needs to automatically print to a printer. It needs to have options to provide the IP address of the remote computer, in addition to setting the delay on the shots, and whether or not the program will save copies of the photos to disk. The Photo Mode needs to gather information on the user so that their photos can be looked up. The Printer Mode needs to query information from a Photo Mode instance. Finally, the application needs to have the availability to select the number of shots that will be taken. Problem Domain Utilizing a Photo Booth at ones wedding is a popular affair in this day and age. Many who have done this before recommend using Photo Booth for Apple OS X; however this leaves the majority of us with PCs unable to enjoy the benefits of an easy to use photo booth application. Although there are other applications, such as Photo Boof, I hope that the ease of access of my application, as well as the additional features that it includes will tap into the niche market that photo booths at weddings provide.

Literature Search Connecting a webcam to a win32 C++ application in Visual Studio 2010 first requires one to link the proper libraries; the tutorial by snoopy11 at dreamincode.net specifically mentions the vfw32.lib and the gdi32.lib. The next components that the tutorial details is an overview of the functions it needs, these functions include a standard function for window control, and two additional functions which handle the bitmap processing, and saving of the bitmap to disk. What was of particular use is how they connect to the webcam through use of the Video for Window library. The tutorial first creates a capture window which will contain the content that is pulled from the webcam. Once the capture window is created, the tutorial then makes use of the SendMessage() function. The Video for Windows library provides control for an address space that corresponds to different commands that one can perform with the webcam. The SendMessage() function works by directing a message from the capture window to the windows Kernel while also appending the address of the particular command you wish to perform. For example SendMessage(camhwnd,WM_CAP_DRIVER_CONNECT,0,0) connects the driver to the capture window camhwnd. Finally the tutorial shows you how to grab an image from the webcam by copying the bitmap contents of a webcam frame to the clipboard and then processing that into a Bitmap Handle. The fundamentals that this tutorial provided are still prevalent in my project as it is an efficient way to interface with the webcam. Solution Taking what I had learned in that tutorial, and continuing my C++ education past the scope of the book Ivor Horton's Beginning Visual C++ 2010 I began a prototype of my project. After my readings, I had predetermined that it was in my best interest to use managed C++ and utilize the Common Language Runtime available in the .NET framework. The managed C++ language seemed appropriate as it helps out with aspects of C++ that may be difficult for newcomers to the language, such as garbage collection. Visual Studio 2010 also provides a tool for building GUIs with the .NET framework, and I had it fixed in my head that this was going to be my approach for my application. Unfortunately, one cannot easily use unmanaged code in a managed code environment, and therefore I abandoned the idea of using the .NET framework and decided to build the GUI in the same way that it was done in the tutorial. This is why my projects GUI was built using the Win32 API.

The way the Win32 API builds a GUI is through function calls that create a HWND, or a Window Handle. In the process of creating a window, a pointer to the window procedure must be passed to the WNDCLASS structure that is used. This pointer essentially represents a function that corresponds with the window. The final overhead concept of Win32 API GUIs is the message pump. Programming in Windows relies heavily on the idea of messages, messages are what are sent to the Windows Kernel, and messages that are available for applications are returned to use. This is important in building a GUI because each user interaction, a click of the mouse, or keyboard press, has a message being sent. Therefore the main function, WinMain(), must use what is called a message pump to see if available messages that are being passed through the application need to be dispatched out. If a messages destination is one of the windows that correspond to my application, then that message will be passed to the appropriate window procedure automatically. This is why there are three separate window procedures in my program, EasyModeProc(), PhotoModeProc(), and PrinterModeProc(), because my GUI is made up of three separate Window Handles. In the Win32 API it is the programmers responsibility to define actions for system messages, WM_CREATE, WM_DESTROY, and WM_COMMAND. WM_CREATE is the message that is passed when the window is first created; this is where the programmer defines buttons and menu items that will be present in the GUI, it also maps these buttons to a HMENU that is used later when commands are passed. WM_DESTROY is the message that is sent when a user closes the window; the programmer uses this to clean up resources used in the GUI. Since my GUI only has one window active at a time, this is also where the WM_QUIT message is sent to the message queue so that WinMain() knows to break out of the message loop, this is done through the use of the function PostQuitMessage(0). Finally, and most important for the behavior of the application is the WM_COMMAND message, this message is passed through the Window Process when an action is performed within the window. Each action passes this message to the queue and also attaches an HMENU to the WPARAM of the function call. It is the programmers responsibility to decipher the HMENU item and perform appropriate actions based on this. When I first began working on my project, it had been my intention to attempt an object oriented approach to the solution. I began by creating a class Mode that corresponded with each Mode that the application can run in; in the constructor of Mode is where I destined that the

window would be created. However, as mentioned before in order to create a window a pointer to a windows procedure needs to be provided. Since this address needs to be known this would have to be a static method. Since the messages are handled in the windows procedure, functions called in the windows procedure would also need to be static as well. This is why I made the design decisions that you see in the final application. The class Mode still is where the window is created, but the window procedure that it is associated with is a global function. This is true for most of the functions that were subsequently written. For the most part the remainder of the application was written using global functions, but there are a couple of other classes that help out with additional tasks, such as a linked-list data structure class that is used to store the database items, and a wrapper class that handles the network components. After laying out the framework of the GUI structure, it was important to have a way to handle the bitmaps that were being captured other than simply saving them to disk. This is when I discovered the GDI+ library which enables applications to use graphics and formatted text on both the video display and the printer (MSDN). This library enables me to not only modify bitmaps, arrange multiple bitmaps together, and print bitmaps; it also allows me to draw images to the GUI. Very early on in development, after producing my first prototype, I realized that I would need to develop a multithreaded application. It is essential that there be a delay between each shot that is taken, and the easiest way to produce a delay is by using the Sleep() function. Unfortunately if the application is asleep it no longer updates the preview window for the webcam, and the GUI becomes unresponsive. The solution is to release the actions that need to be performed while shots are taken to a separate thread. By placing these actions in a separate thread the application appropriately delays a number of seconds before taking each shot. With the delay between each shot set up, it was difficult to know when the shot was being taken. It was necessary to develop a countdown timer that shows the user the delay as it is actually happening. My first instinct to developing a countdown timer was to display bitmaps using the GDI+ library over the top of the capture window. Unfortunately there is no way to simply create an overlay, and anything placed over the capture window would disappear when it redraws multiple times per second in order to show a preview. I learned that using the Video for Windows library you can have a callback function that is called every time a capture takes place, so conceptually I thought that I could have this function draw the appropriate bitmap over

the preview image. In practice, however, this didnt work as the image displayed was being refreshed too rapidly to produce a clean image. It was decided then that the countdown timer needed to be moved outside the Capture Window, and should have its own child window next to the GUI buttons. In the end I prefer having a separate countdown timer, as opposed to drawing it over the capture window, it enhances the GUI. When first designing my application, I had anticipated that when the Photo Mode was taking shots for users it would keep copies of the bitmap taken in memory. This hadnt crossed my mind as an issue until after I built the first prototype of my application; it was then that I realized that each bitmap in memory can take over a megabyte of memory. If Photo Mode was to be running for a long time, and hosting a large number of users, the application could easily use more memory than what would be acceptable for this application. Therefore it is necessary to save files that are needed to disk, and there needs to be an appropriate way to look up these files when they are needed to deliver to the Printer Mode. I realized that easiest way to accomplish this would be to create a linked-list of nodes, where each node contains the users first name, last name, and a string to the file path. When a query is placed for a users name, the Photo Mode client looks up the file path and loads the bitmap into memory. Once the bitmaps are loaded into memory they are then transmitted to the Photo Mode. In order for the bitmaps to be transmitted without error between clients, a reliable transport protocol would need to be used. That is why I decided to use TCP to transmit files between clients running my application, and have communication taking place on port 20777. Conceptually, the Printer Mode client queries a username that they wish to contact in the form of query firstName lastName. The Photo Mode responds to queries in this format by first letting the Printer Mode know how many photos are available, and then it proceeds to transfer these files to the Printer Mode client. Abstractly, these are all of the components that make up the final application. Without any one of these concepts the final application would not behave in the same fashion that it currently does. The most difficult aspect of achieving these solutions was not in the design of the components; it was in figuring out how to get the Windows API, and the linked libraries to achieve the behavior that I wanted it to. Results

The final results of my application still resemble what I had initially set out to accomplish, the only noticeable difference is the exclusion of an editing feature. Theoretically this could be done by using the features available in the GDI+ library, which include effects such as red-eye removal and cropping. There is also the opencv library which performs additional special effects. Unfortunately, I didnt have time to develop the additional window that would include the editing, so this is one feature that regrettably was excluded. The final version of the Easy Mode allows users to select the number of shots that they want to take, and doesnt confuse the user with a large amount of options or other information. The Photo Mode more or less operates in a similar fashion to the Easy Mode, however instead of the number of shots this mode takes in the first and last name fields. Finally, Printer Mode was revised slightly from my initial thoughts, instead of showing the user all of the shots that are available in a thumbnail view; Printer Mode allows the user to cycle through each shot available and select ones that they want for printing. Samples of each of the User Interfaces are shown below:

When I was creating my first prototype, and first using the GDI+ library, I was a novice at coding in C++. Without ample experience I didnt know why I couldnt have a method that returned type Bitmap. I was able to get the method to work properly, but only if I returned a Bitmap pointer instead of a Bitmap object. I later learned that this is probably because the Bitmap class in the GDI+ library doesnt provide access to the constructor from outside the scope of the class, so whenever I was trying to make a function of type Bitmap the compiler thought I was trying to declare a constructor for Bitmap. The way I was constructing Bitmaps was through a static function called, Bitmap::FromFile(), which creates the Bitmap function from a file. So returning the pointer of the Bitmap that was created with that function call was an appropriate solution.

I also experienced issues while making my application multithreaded. As I discussed earlier, when the application triggers a photo to be taken, a new thread is created. It was easy to create the thread, having a special function that is run once the thread is created. The issue was in cleaning up the thread. Most documentation and examples handles threads that are being created from a Main function or at a point in the code where it is okay to wait for all threads to complete execution. The point in my code where the thread needs to be created is in one of the Windows Procedures, after a button is pressed. This function works to handle the message that are handed to the GUI; to wait for the thread to execute would cause the same issues as before. If I were to create a thread local to that function, and not wait for it to execute, then I would lose the handle to that thread and not be able to clean it up. The best solution for this issue was to create a global handle of the thread that would be instantiated when it is need, as many times as needed. I also created a boolean variable so that the user couldnt overwrite the thread if they were to click the button multiple times. Finally, I created a mutex lock so that there wouldnt be issue when setting the boolean variable. If the program were to exit with the thread still in execution, the cleanup code waits for the thread to finish execution. Once I had these items in place, I was able to have the thread execute exactly how I intended. The network components of my project also provided a source of additional frustration. Initially, I had tried to create these components by directly interfacing with the winsock2 library and handling all communications this way. Using these function calls, I was able to successfully exchange messages between a client and server, however there were bugs that would sometimes erroneously cause a packet to be received wrong at the server end. At the time I also did not have enough understanding of network programming to realize how to send a file through the network. This is when I discovered the WComm class that was written by kbfromindia at codeproject.com. This class wraps the function calls to the winsock2 library, and provides methods fileSend() and fileReceive() which made it very easy to send files through the network at my desired port. Even though I experienced many issues along the way, I am very satisfied with the results that I achieved in this project. The overall application runs very well considering I had no experience in C++, and no experience in programming for Windows prior to this project. Furthermore, when facing the problems I encountered I was able to draft practical solutions in order to get the job accomplished.

Conclusions My work in development of this application embodied a lot more than simply completing a project. I was able to pick up a new programming language, and commit a substantial amount of code (around 2000 lines) in this language in a relatively short timeframe. I was also able to learn how to accomplish a task using real world references, such as the Microsoft Developers Network. Finally, I was able to create a finished product which satisfied the needs of my fiance and me in our future wedding. As a learning experience, I would consider this the most successful aspect of this project. I came into the venture with no experience in C++, and little confidence in potential as a C++ developer. Since then, I would now consider C++ my desired language to work with. I would also consider my proficiency in the language to be much higher having completed this project, then if I had only read Ivor Horton's Beginning Visual C++ 2010. I also learned a lot about tackling a project on my own. I realize now that prototyping was successful in teaching me C++ quickly, but it wasnt successful in helping me to create the best product. If I were to rebuild my project with what I know now, I would no longer use the Win32 API to build my GUI. It proved cumbersome, and a lot more time than what was necessary was spent on building the GUI framework, and making sure it worked. I think I would have much rather have worked with the MFC, or the Microsoft Foundation Class Library, as Visual Studio provides tools to build GUIs for this library quickly. The MFC also has a predefined class structure, so my goal of an objectoriented solution could potentially be achieved. I believe that advantages of my work would be first that I learned an enormous amount. I also think that my work, namely having a separate Photo Mode and Printer Mode, may be an exclusive position because there are no other products that I know of that perform the same functionality. However, there are a couple of disadvantages of my work. First the user interface isnt as elegant as other products, and doesnt provide enough control of the environment to satisfy many users. The user interface is also cumbersome at times, as there is one particular bug which causes the video driver to disconnect. Even though I wasnt satisfied with the effort I spent on generating the GUI for my project, as a finished product I am proud of the work I accomplished. If there was more time, I would like to take what I have completed today as a second prototype and again rebuild my solution. I think that future work should include rebuilding the GUI with the MFC library, and

10

obtaining an object oriented solution. I also think that instead of using the outdated Video for Windows library, future directions should utilize a more recent library, such as Direct Show. Acknowledgements I would like to thank my advisor, Dr. Eric M. Schwartz, for his guidance, suggestions, and encouragement in successful completion of my project. Without his help, my project wouldnt have any of the features that I am most proud of developing. References Horton, Ivor. Ivor Hortons Beginning Visual C++ 2010. Indianapolis: Wiley Publishing, 2010. Print. Kbfromindia. CodeProject. C++ Winsock Client To Server File Transfer Made Easy. CodeProject, 2006. Web. 11 November 2011 < http://www.codeproject.com/KB/cpp/WSFileTransfer.aspx>.

Microsoft Developer Network (MSDN). Microsoft Corporation, 2011. Web. 7 November 2011 <http://msdn.microsoft.com/en-us/>. DreamInCode. Webcam Tutorial." MediaGroup1 LLC, 2010. Web. 2 September 2011 <http://www.dreamincode.net/forums/topic/193519-win32-webcam-program/>. Biography John Paul Swaynos was born in Orlando, Florida on May 16th, 1989. He completed his secondary education at Lake Brantley High School in Altamonte, Florida and is completing his baccalaureate degree in Computer Engineering specializing in Software at the University of Florida (Gainesville, FL), where he expects to graduate on December 17th, 2011. He and his fiance will be getting married after their graduations on June 23rd, 2012 in Orlando, Florida. He has been a longtime member of the computer support team at the Department of Housing at the University of Florida, where his work helps to solve any issue that Housing Staff may have with their computers. After graduation, John hopes to find employment in the Orlando area as a Software Engineer, continuing a lifelong passion of computers and technology.