CSE 403: Software Engineering, Spring 2015
courses.cs.washington.edu/courses/cse403/15sp/
The Joel Test:
12 Steps to Better Code
Emina Torlak
[email protected]
Outline
1. Do you use source control?
2. Can you make a build in one step?
3. Do you make daily builds?
4. Do you have a bug database?
5. Do you fix bugs before writing new code?
6. Do you have an up-to-date schedule?
7. Do you have a spec? www.joelonsoftware.com/
8. Do you have quiet working conditions? articles/fog0000000043.html
9. Do you use the best tools money can buy?
10. Do you have testers as part of the team?
11. Do you have interview candidates write code?
12. Do you do hallway usability testing?
2
Outline
1. Do you use source control?
2. Can you make a build in one step?
3. Do you make daily builds?
Score <= 10 usually
4. Do you have a bug database?
means your team is
5. Do you fix bugs before writing new code? in trouble!
6. Do you have an up-to-date schedule?
7. Do you have a spec?
8. Do you have quiet working conditions?
9. Do you use the best tools money can buy?
10. Do you have testers as part of the team?
11. Do you have interview candidates write code?
12. Do you do hallway usability testing?
2
Outline
1. Do you use source control?
2. Can you make a build in one step?
3. Do you make daily builds?
Score <= 108 usually
4. Do you have a bug database?
means your team is
5. Do you fix bugs before writing new code? in trouble!
6. Do you have an up-to-date schedule?
7. Do you have a spec?
8. Do you have quiet working conditions?
9. Do you use the best tools money can buy?
10. Do you have testers as part of the team?
11. Do you have interview candidates write code?
12. Do you do hallway usability testing?
2
Do you use source control?
3
Do you use source control?
• Source control …
• allows multiple developers to collaborate
• keeps project in consistent state
• tracks changes and enable roll-back
• manages multiple versions
• saves data in case of a disaster
• is the authoritative source for “daily build”
3
Do you use source control?
• Source control …
• allows multiple developers to collaborate
• keeps project in consistent state
• tracks changes and enable roll-back
• manages multiple versions
• saves data in case of a disaster
• is the authoritative source for “daily build”
The ZFR should indicate the state of your repository.
3
Can you make a build in one step?
4
Can you make a build in one step?
• A single script that
• [does a full checkout from scratch]
• rebuilds every line of code
• makes the binary executable files in all versions,
languages and #ifdef combinations
• [creates the installation package]
• [creates the final media, web site, …]
4
Can you make a build in one step?
• A single script that
• [does a full checkout from scratch]
• rebuilds every line of code
• makes the binary executable files in all versions,
languages and #ifdef combinations
• [creates the installation package]
• [creates the final media, web site, …]
• All steps are automated and exercised regularly
4
Can you make a build in one step?
• A single script that
• [does a full checkout from scratch]
• rebuilds every line of code
• makes the binary executable files in all versions,
languages and #ifdef combinations
• [creates the installation package]
• [creates the final media, web site, …]
• All steps are automated and exercised regularly
• So, why is this valuable?
4
Do you make daily builds?
5
Do you make daily builds?
• Build the entire product every day and run a good
test suite against the new version
• build from checked in sources
• automatic and frequent
5
Do you make daily builds?
• Build the entire product every day and run a good
test suite against the new version
• build from checked in sources
• automatic and frequent
• Goal: find out early that you’ve got problems and
fix them before disaster strikes
5
Do you make daily builds?
• Build the entire product every day and run a good
test suite against the new version
• build from checked in sources
• automatic and frequent
• Goal: find out early that you’ve got problems and
fix them before disaster strikes
• Benefits
• minimizes integration risk
• reduces risk of low quality
• supports easier defect diagnosis
• improves morale for developers, managers, customers
5
Do you make daily builds?
• Build the entire product every day and run a good
test suite against the new version
• build from checked in sources
• automatic and frequent
• Goal: find out early that you’ve got problems and
fix them before disaster strikes
• Benefits
• minimizes integration risk
• reduces risk of low quality
• supports easier defect diagnosis
• improves morale for developers, managers, customers
The ZFR must include your build script or sequence.
5
Do you have a bug database?
6
Do you have a bug database?
• You can’t keep the bug list in your head
• especially with multiple developers and multiple customers
6
Do you have a bug database?
• You can’t keep the bug list in your head
• especially with multiple developers and multiple customers
• Moreover, looking at the history of bugs can be insightful!
6
Do you have a bug database?
• You can’t keep the bug list in your head
• especially with multiple developers and multiple customers
• Moreover, looking at the history of bugs can be insightful!
• To characterize a bug consider:
• how to reproduce it
• expected behavior, actual behavior
• responsible party, status, priority
6
Do you have a bug database?
• You can’t keep the bug list in your head
• especially with multiple developers and multiple customers
• Moreover, looking at the history of bugs can be insightful!
• To characterize a bug consider:
• how to reproduce it
• expected behavior, actual behavior
• responsible party, status, priority
• Best to use what is integrated with your code hosting.
6
Do you have a bug database?
• You can’t keep the bug list in your head
• especially with multiple developers and multiple customers
• Moreover, looking at the history of bugs can be insightful!
• To characterize a bug consider:
• how to reproduce it
• expected behavior, actual behavior
• responsible party, status, priority
• Best to use what is integrated with your code hosting.
For the beta release assignment, we’ll
be asking to see a log of your bugs.
6
Do you fix bugs before writing new code?
7
Do you fix bugs before writing new code?
• Why not fix them later?
7
Do you fix bugs before writing new code?
• Why not fix them later?
• Familiar with the code now
7
Do you fix bugs before writing new code?
• Why not fix them later?
• Familiar with the code now
• Harder to find (and fix) later
7
Do you fix bugs before writing new code?
• Why not fix them later?
• Familiar with the code now
• Harder to find (and fix) later
• Later code may depend on this code (try building on
quicksand…)
7
Do you fix bugs before writing new code?
• Why not fix them later?
• Familiar with the code now
• Harder to find (and fix) later
• Later code may depend on this code (try building on
quicksand…)
• Bugs may reveal fundamental problems
7
Do you fix bugs before writing new code?
• Why not fix them later?
• Familiar with the code now
• Harder to find (and fix) later
• Later code may depend on this code (try building on
quicksand…)
• Bugs may reveal fundamental problems
• Leaving all bugs to the end will make it harder to
understand and keep the schedule
7
Do you have an up-to-date schedule?
8
Do you have an up-to-date schedule?
• Keeps expectations realistic
• For the team, customers, stakeholders
8
Do you have an up-to-date schedule?
• Keeps expectations realistic
• For the team, customers, stakeholders
• Allows for more accuracy
• Use experience to improve estimates
8
Do you have an up-to-date schedule?
• Keeps expectations realistic
• For the team, customers, stakeholders
• Allows for more accuracy
• Use experience to improve estimates
• Helps prevent feature creep
• Don’t take on anything without checking the schedule first
8
Do you have an up-to-date schedule?
• Keeps expectations realistic
• For the team, customers, stakeholders
• Allows for more accuracy
• Use experience to improve estimates
• Helps prevent feature creep
• Don’t take on anything without checking the schedule first
For the SDS, we asked for a schedule.
For later releases, we ask you to
highlight any changes, and keep all
documents up to date.
8
Do you have a spec?
9
Do you have a spec?
• Easier to fix problems at the design stage
9
Do you have a spec?
• Easier to fix problems at the design stage
• You know what you are trying to build
• So do your teammates and customer
9
Do you have a spec?
• Easier to fix problems at the design stage
• You know what you are trying to build
• So do your teammates and customer
• More likely that you build the right thing
• Pieces fit together
• Customer is satisfied
9
Do you have a spec?
• Easier to fix problems at the design stage
• You know what you are trying to build
• So do your teammates and customer
• More likely that you build the right thing
• Pieces fit together
• Customer is satisfied
• Conceptual integrity for your project
9
Do you have a spec?
• Easier to fix problems at the design stage
• You know what you are trying to build
• So do your teammates and customer
• More likely that you build the right thing
• Pieces fit together
• Customer is satisfied
• Conceptual integrity for your project
• Undocumented code has low value
• Hard to maintain and to extend
• Hard to bring new developers on board
9
Do you have hallway usability testing?
10
Do you have hallway usability testing?
• Grab someone in the hallway and make them
use your code
10
Do you have hallway usability testing?
• Grab someone in the hallway and make them
use your code
• Key idea: get feedback fast
10
Do you have hallway usability testing?
• Grab someone in the hallway and make them
use your code
• Key idea: get feedback fast
• A little feedback now ≫ lots of feedback later
10
Do you have hallway usability testing?
• Grab someone in the hallway and make them
use your code
• Key idea: get feedback fast
• A little feedback now ≫ lots of feedback later
• You will get most of the valuable feedback
from the first few users
10
Joel’s disclaimer
11
Joel’s disclaimer
• These are not the only factors that determine success or failure
• A great team will not help if you are building a product no one wants
• An incredibly talented team might produce an incredible product
without these guidelines
11
Joel’s disclaimer
• These are not the only factors that determine success or failure
• A great team will not help if you are building a product no one wants
• An incredibly talented team might produce an incredible product
without these guidelines
• But all things being equal, these factors indicate a disciplined
team that can consistently deliver
11
Joel’s disclaimer
• These are not the only factors that determine success or failure
• A great team will not help if you are building a product no one wants
• An incredibly talented team might produce an incredible product
without these guidelines
• But all things being equal, these factors indicate a disciplined
team that can consistently deliver
“The bummer about The Joel Test is that you
really shouldn't use it to make sure that your
nuclear power plant software is safe.”
11
Making mission-critical software safe …
Gerard J. Holzmann.
Mars code, 2014.
12
Making mission-critical software safe …
• First, standard precautions for reducing risk in Gerard J. Holzmann.
complex software systems: Mars code, 2014.
12
Making mission-critical software safe …
• First, standard precautions for reducing risk in Gerard J. Holzmann.
complex software systems: Mars code, 2014.
• A good software architecture with a clean
separation of concerns, data hiding, modularity,
well-defined interfaces, and strong fault-
protection mechanisms.
12
Making mission-critical software safe …
• First, standard precautions for reducing risk in Gerard J. Holzmann.
complex software systems: Mars code, 2014.
• A good software architecture with a clean
separation of concerns, data hiding, modularity,
well-defined interfaces, and strong fault-
protection mechanisms.
• A good development process, with clearly stated
requirements, requirements tracking, daily
integration builds, rigorous unit and integration
testing, and extensive simulation.
12
Making mission-critical software safe …
Gerard J. Holzmann.
Mars code, 2014.
13
Making mission-critical software safe …
• Risk-based coding rules
• Six compliance levels. Gerard J. Holzmann.
• Level 3: “We require that the flight software as a Mars code, 2014.
whole, and each module within it, had to reach a
minimal assertion density of 2%.”
13
Making mission-critical software safe …
• Risk-based coding rules
• Six compliance levels. Gerard J. Holzmann.
• Level 3: “We require that the flight software as a Mars code, 2014.
whole, and each module within it, had to reach a
minimal assertion density of 2%.”
• Tool-based code review
• Peer review great at discovering design flaws.
• But tools are better at discover coding flaws.
13
Making mission-critical software safe …
• Risk-based coding rules
• Six compliance levels. Gerard J. Holzmann.
• Level 3: “We require that the flight software as a Mars code, 2014.
whole, and each module within it, had to reach a
minimal assertion density of 2%.”
• Tool-based code review
• Peer review great at discovering design flaws.
• But tools are better at discover coding flaws.
• Formal methods
• Used for critical software and hardware components.
• Provides high assurance but requires expertise, time.
13
Summary
The Joel Test for 403:
1. Do you use source control?
2. Can you make a build in one step?
3. Do you make daily builds?
4. Do you have a bug database?
5. Do you fix bugs before writing new code?
www.joelonsoftware.com/
6. Do you have an up-to-date schedule? articles/fog0000000043.html
7. Do you have a spec?
8. Do you do hallway usability testing?
14
