FinalProject Checkpoint4
FinalProject Checkpoint4
- Checkpoint 4
Checkpoint Description
So far, this project has been very heavy on concepts revolving around graphics, and object-oriented programming. This is the checkpoint
where we are going to start applying knowledge from this particular course. This time, we are going to be focusing in pretty hard on two
particular subjects –
We’re going to rely on these topics to add the ability to both parse and load bitmap images into our program.
By the end, you too will be able to have your own Cactus Jack in your Edit Window
Objectives
1. Objective 1
2. Objective 2
3. Objective 3
Objective 1
Objective checklist:
• Fix up our ability to draw doodles
o Get rid of the pixel-gaps in our doodles
o Make the lines that we draw thicker
1.0 Fixing our Doodles
If we run our doodling program right now, and try to create a doodle inside of it, we’ll likely end up a little disappointed by the results we get.
My best attempt at drawing a smiley face
Remember back at the beginning of this project when I said that all programs can only “refresh” themselves so often? In our case, our program only
“refreshes” 60 times per second. We know this because of this line in DesktopLauncher.java
But we’re still not getting “perfect” results. There are still lots of annoying and gross looking gaps between all of our pixels. This is happening
because our mouse is capable of moving far faster than our program can ever “refresh”. The image below illustrates the problem
Our mouse moving over multiple pixels before the program has a chance to refresh
Because our program only colors in whatever pixel is CURRENTLY moused over, we can end up skipping a lot of pixels along the way. We could fix
this ourselves, but LibGDX has a nice way of doing this for us. We can use a new method to help us out here.
Hop into EditWindow.Java and change the following code.
private void paintAtPosition(Vector2 worldPosition) {
_doodleMap.drawLine(0, 0, 300, 300);
_doodleMap.drawPixel((int) (worldPosition.x - Position.x), (int) (Scale.y - worldPosition.y));
Now, when we click, we won’t be drawing where our mouse is pointing just yet, but notice how we draw a perfect FULL line across the screen
A perfect line with no skipped or missing pixels
Let’s use .drawLine() to make our program fill in the gaps between the pixel we were JUST BARELY mousing over, and the pixel we are CURRENTLY
mousing over.
public class EditWindow extends Rec2D implements IClickable{
private Vector2 _previousPaintPosition;
private void paintAtPosition(Vector2 worldPosition) {
_doodleMap.drawLine(0, 0, 300, 300);
Vector2 paintPosition = new Vector2(worldPosition.x - Position.x,Scale.y - worldPosition.y);
_doodleMap.drawLine(
(int) _previousPaintPosition.x, (int) _previousPaintPosition.y,
(int) paintPosition.x, (int) paintPosition.y);
_previousPaintPosition = paintPosition;
DoodleTexture = new Texture(_doodleMap);
}
If we run this code as-is though, it’ll crash due to an annoying error where _previousPaintPosition is null at the start of our program. Let’s fix this
really quick before testing.
public void onClickDown(Vector2 clickPosition) {
if(_previousPaintPosition == null)
_previousPaintPosition = new Vector2(clickPosition.x - Position.x,Scale.y - clickPosition.y);
paintAtPosition(clickPosition);
}
Making sure _previousPaintPosition ALWAYS has a value
However, we do have on small issue still. If we let go of our mouse, then click somewhere else, our line will JUMP from where we “unclicked” to
where we “clicked”
Drawing a line, unclicking, then drawing another line starting from the red circle
To fix this, all we have to do is add a bit of code to our onClickUp() method
Instead of ONE click drawing ONE line, a pixel thick, we’ll make ONE click draw a VARIETY of lines all offset from each other.
private void paintAtPosition(Vector2 worldPosition) {
Vector2 paintPosition = new Vector2(worldPosition.x - Position.x,Scale.y - worldPosition.y);
int startX = (int) _previousPaintPosition.x;
int startY = (int)_previousPaintPosition.y;
int endX = (int) paintPosition.x;
int endY = (int) paintPosition.y;
_doodleMap.drawLine(startX, startY, endX, endY);
_doodleMap.drawLine(startX + 1, startY, endX + 1, endY);
_doodleMap.drawLine(startX - 1, startY, endX - 1, endY);
_doodleMap.drawLine(
(int) _previousPaintPosition.x, (int) _previousPaintPosition.y,
(int) paintPosition.x, (int) paintPosition.y);
Our lines are thicker because we draw to the left and right of the clicked pixels
Objective 2
Checklist:
• Understand the motivation behind us choosing to use the .bmp image format
2.0 Introduction to Image Formats
Our application’s final purpose is to load images, let us edit them, and then save them back to a file with all of our changes. Doing this will require
an understanding of image file structure. Think for a moment on the multitude of different image file-formats that you’ve seen in your life.
If all of these formats solve the same task, storing image data, why are there so many different types?
Consider that the latest IPhone has a camera that shoots at 48-Megapixels. A Megapixel is a term used to represent 1,000,000 (One MILLION) pixels.
That means, an IPhone can take an image that contains 48 MILLION pixels. Recall that a pixel is just a container for color data, and colors are
represented by their red, green, blue, and alpha components.
If we use integers to represent red, green, blue , and alpha that means four ints make up a single pixel. In other terms, that would mean -
1 Pixel = 16 Bytes
1 MP = 16 Million Bytes
Or
1 MP = 16 Megabytes
768 Megabytes
That’s almost an entire Gigabyte! For reference, most images range from 1-30 Megabytes, DRAMATICALLY less than the number we just came up
with.
The reason why so many image formats exist is primarily to provide a sliding scale in terms of quality and storage space for users.
A png for example, applies heavy compression to an image in order to store it in a relatively small amount of space
A RAW image however, will store data without attempting to compress it, providing higher visual quality, at the cost of MASSIVELY increased storage
requirements.
For this project, we’ll be working with .bmp files. This format was chosen primarily due to the fact that it applies ZERO compression to images
(which will make our job MUCH easier) and because it’s a moderately simple and popular format to work with.
In the Checkpoint 4 assignment, there are a number of image files. If you haven’t already, download those now.
Every file format in existence, holds some kind of file header. If you’ve never seen one before, the header for a bmp might look scary at first, but for
our purposes, will be much simpler than it might seem on the surface.
Chart can be found here -
https://www.ece.ualberta.ca/~elliott/ee552/studentAppNotes/2003_w/misc/bmp_file_format/bmp_file_format.htm
We’ll be referring to this table regularly, so feel free to save it or open it up on another monitor (if you have one)
Objective 3
Checklist:
• Load a bitmap image into our application so we can doodle over it
• Create utility methods that allow us to easily work around Java’s limited byte type
3.0 ImageInputOutput.java
Loading images into (and eventually back out of) our program will be a bit of a complicated task. To do this, let’s create a new class
ImageInputOutput.java and start by turning it into a Singleton.
public class ImageInputOutput {
public static ImageInputOutput Instance;
public ImageInputOutput() {
Instance = this;
}
}
Let’s also add a method to this class called loadImage() that takes a file-path (A String) as an argument.
public void loadImage(String filePath) {
System.out.println("I'm going to load " + filePath);
}
Let’s also make sure to call loadImage from somewhere in our program. The top of our create method inside seems like a fine enough place for now.
public void create () {
Instance = this;
new ImageInputOutput();
ImageInputOutput.Instance.loadImage("FakeFilepath");
The third line looks funny, but is technically valid code. Maybe don’t use this in your own programs going forwards, but I think it’s fun just
to know that you can get away with this sort of thing, so I’m keeping it for pure novelty
Run your code, and you should see a print statement verifying that our method is being called.
Woo!
ImageInputOutput.Instance.loadImage("blackbuck.bmp");
We can now attempt to read in our file with the following code given to us by LibGDX
public void loadImage(String filePath) {
byte[] bytes = Gdx.files.internal(filePath).readBytes();
System.out.println("Loading file of size " + bytes.length);
}
Reading in the file, saving the contents to an array, then printing the length of that array
We can confirm that this worked by running and checking the console output against the size of the file, as read by our operating system.
Now, before moving forwards, I want to point out a couple of interesting and important items.
We’ll come back to these points as we go through the project, but I want to expand a little on them here.
Why are we using some library from LibGDX instead of a standard Java library?
LibGDX actually uses an InputStream itself, but by using the LibGDX version of FileIO, we can easily talk between projects. R ecall that
ImageEditor-core and ImageEditor are two entirely separate projects. We also have a vague, but general assurance that however libGDX
accomplishes this task, it will do it in a relatively speedy manner
I also want to pose the question – If some of the information in our file is actually made of integers (4 bytes), why are we reading everything in as
just bytes? Why not use a nicer library like Scanner, where we can differentiate the types of data we’re reading in?
Because these approaches make POOR USE of our CPU and memory hardware!
Recall that small buffer sizes, and “magical” data parsing are both incredibly expensive operations! We performed a demonstration showcasing this
in LTSDemo.java back in Section 7.
If the theme of this class is hardware utilization, we can MUCH BETTER utilize our hardware by reading everything in as simple bytes, then piecing
back together data as needed.
We can do this super easily by checking each of the first two bytes in our array
public void loadImage(String filePath) {
byte[] bytes = Gdx.files.internal(filePath).readBytes();
if(bytes[0] != 'B' || bytes[1] != 'M') {
System.out.println(filePath + " is NOT a bitmap image");
return;
}
If you now try to load another file type, your code will refuse to parse the image, and exit the loadImage method.
The next piece of data that we’ll need however, will prove to be a bit more complicated to parse
This is because it is given as a series of 4 bytes, that make up an integer value. Before moving forwards, let’s try to confirm what the value of FileSize
is supposed to be, before attempting to parse it out.
https://hexed.it/
If using the above link, we can simply drag and drop our image into our webpage, and the center view should populate with data from our image. It
might look a bit intimidating, but I’ll break down the section that will be important to us going forwards.
If we take a look, we should be able to see that the first TWO BYTES of our file represent the ascii characters for BM by taking a look at section 4. If
we now select the third byte in our file (by clicking), and read from section 2, we can try to figure out how big the file is supposed to be.
And, as expected, this number matches the expected result! However, note that the values for 32-bit and 24-bit remain the same. The exact details
of this go slightly beyond the scope of this project, but the reason for this is that byte-data in a bitmap image is stored in LITTLE-ENDIAN form. The
curious minded can read below for what that means.
https://en.wikipedia.org/wiki/Endianness
Also note that our first byte can also be parsed as the number 54.
Alright, now we can read in the size of the file very simply, using the following code.
public void loadImage(String filePath) {
byte[] bytes = Gdx.files.internal(filePath).readBytes();
if(bytes[0] != 'B' || bytes[1] != 'M') {
System.out.println(filePath + " is NOT a bitmap image");
return;
}
byte[] fileSize = {bytes[2], bytes[3], bytes[4], bytes[5]};
System.out.println("The size of the file is " +
fileSize[0] + " " + fileSize[1] + " " + fileSize[2] + " " + fileSize[3]);
We can confirm that each of these individual bytes is correct by checking them against the values in our hex-editor.
If we reverse the order of our individual bytes, and concatenate (smash) them together, we get the 4-byte binary number that represents 786,486!
So, all we have to do now is do that!
…..somehow
This time, just for the purpose of demonstration, instead of making Util a Singleton, like we have been, we’ll just manually mark our methods inside
as static.
Let’s also call our method, and send it some dummy data to work with so we can test our code as we go.
byte[] fileSize = {bytes[2], bytes[3], bytes[4], bytes[5]};
byte[] dummyData = {5, 10, 15, 20};
System.out.println("The size of the file is " + Util.bytesToInt(dummyData));
System.out.println("The size of the file is " +
fileSize[0] + " " + fileSize[1] + " " + fileSize[2] + " " + fileSize[3]);
Calling bytesToInt
Based on our current understanding, the values in dummyData have the following values.
5 = 00000101
10 = 00001010
15 = 00001111
20 = 00010100
If we reconstruct this binary data backwards, we can find the expected integer that these bytes represent.
Throwing this data into a simple binary calculator tells us that our array of bytes represents the integer value of –
336,529,925
However, this approach is WILDY unperformant, as it requires a large number of String operations.
This is because our bytes don’t have their BINARY data stored into the string, but their DECIMAL NUMERAL data.
So how do we do it?
This is where we use an operation that many of you may never have used before, the bit-shift operation “<<”
Let’s start by first casting all of our bytes to ints
public static int bytesToInt(byte[] bytes) {
int first = bytes[0];
int second = bytes[1];
int third = bytes[2];
int fourth = bytes[3];
int result = 0;
It is now recommended that you begin following along again
However, it’s not immediately obvious how we might be able to achieve this operation ourselves. Instead, consider that we can “recreate” this
general behavior by using arithmetic instead by performing the following operations.
00010100 00000000 00000000 00000000
00000000 00001111 00000000 00000000
00000000 00000000 00001010 00000000
+00000000 00000000 00000000 00000101
= 00010100 00001111 00001010 00000101
That’s cool and all, but how am I supposed to get my numbers to look like that?
This is where bit shifting comes into play! Let’s take our variable last for example.
And if we shift by an even greater number of bits, say for example, 24 bits
Well then, we’d have something that looks EXACTLY like what we wanted!
00101000 00000000 00000000 00000000
Then, all we’d have to do is add these numbers together, and we’ll have completed our conversion! Below is a scalable version of the code that can
do this, in case we’re ever given 1, 2, 3, or 4 bytes in our array.
Now we can get rid of our dummy data, and try running this on our actual fileSize array
byte[] fileSize = {bytes[2], bytes[3], bytes[4], bytes[5]};
byte[] dummyData = {5, 10, 15, 20};
System.out.println("The size of the file is " + Util.bytesToInt(dummyData));
System.out.println("The size of the file is " + Util.bytesToInt(fileSize));
WOOHOO!
3.4 Reading Color Data
With this hurdle out of the way, now we can go ahead and attempt to parse out any other relevant information from the file header so that we can
start loading in actual Color data!
Now, to avoid getting a little TOO long in the tooth again, here are the following pieces of data that we’ll need.
We can now use these variables to create a new Pixmap with our width and height variables. We can also loop through our array of byte
information (starting from an offset at startPoint, and going until we hit fileSize)
}
A loop that will go through every pixel in our file
We can then perform the following in ImageEditor.java to see if we performed the correct operations.
Pixmap editMap = ImageInputOutput.Instance.loadImage("blackbuck.bmp");
…
Vector2 editWindowSize = new Vector2(500, ScreenSize.y - 50);
_editWindow = new EditWindow(editWindowSize, new Vector2(ScreenSize.x - editWindowSize.x, 0), Color.GRAY);
…
_editWindow.DoodleTexture = new Texture(editMap);
RATS!
Not only are our colors completely messed up, our image is also UPSIDE DOWN
I previously mentioned that each byte in our file has a value ranging from 0-256. This makes logical, intuitive sense, as 256 is the largest number you
can represent with 8-bits.
11111111 = 256
Sigh…
This means that all of that awesome color data inside our bitmap, that ranges from 0-256, is getting completely misinterpreted and mangled by our
program.
So what do we do?
We manually unsign every byte we read from our file.
There are multiple ways of doing this, but I’m going to just turn our byte[] into an int[] because frankly, I’m rather annoyed. Hop into Util.java, and
add the following method.
}
return ints;
}
Now, if we are willing to work with the fact that everything is an integer now, we can do some simple math to “correct” all of our negative numbers.
Our bytes only become negative once they have run out of space, and overflow. With some simple research, we can find out at which point this
happens.
So, if a byte is between the range of 0-127, then it was interpreted properly. This means a value of -128 should actually be 128
-127 should be 129
-126 should be 130
Et cetera.
Once you’ve filled out your method, perform the following in loadImage
• Create a new array of type int, called ints by calling unsignBytes on bytes at the top of your
method
• Instead of looping through bytes at the end, loop through ints
o Replace all references to bytes in your loop with your integer array
• Change the type of r,g, and b to int
• Ints and bytes are a little bit undescriptive
o Let’s rename them to fileBytes and fileIntData
o You can do this easily by right-clicking on the current variable name -> Refactor -> Rename
• Lastly, the rgb color values in our Bitmap are actually stored in backwards order bgr
o Rework the assignment of your rgb values to go in reverse order
3.6 Testing
If you open your program and see the following, then your program is correct
Attempting to draw over the image will cause our image to disappear, but THIS IS FINE FOR NOW
Whew! While this checkpoint was a little more hand-holding than the others, it presented some very challenging technical problems. By making it
this far, you might be surprised to know that we have cleared the last-largest hurdle in creating our image editor.
Next checkpoint, we will work on fixing the current issue that’s causing our buck to disappear once we start drawing, and we’ll also work in the
ability to save images back out to long-term storage.
And once that’s done, we can go on a final victory lap, and use some of the forgotten features we’ve built (I’m looking at you, Button), to kit out and
expand the functionality of our editor massively, without having to write a significant amount of code.