Your browser doesn't support the features required by impress.js, so you are presented with a simplified version of this presentation.

For the best experience please use the latest Chrome, Safari or Firefox browser.

Agenda

Why? Motivation behind the project
A Story and a New Metaphor
Lessons Learned
BIRD Quality Portal

Introduction: why BIRD Quality Portal

  • Inspired by Oita’s question: “How can we find the hot potato defects?” (February 2012)

Gartner 2014
“Furthermore, customers scored SAS’s product quality and support below the survey average. Replacing SAS’s multiple power-user-oriented tools with Visual Analytics should help to improve customers’ perception of SAS’s ease of use, but improving product quality and support must become SAS’s imperative.”

Jennifer Marchi: “AGAIN: 2014 MUST be the year of Customer Satisfaction and References. With customer endorsement of SAS Visual Analytics in 2014, we will strengthen our winning position and demonstrate our ability to execute.​​ It is critical that we have happy customers willing to talk to Gartner and participate in their surveys. “

Story: John Snow

  • Joke: not Jon Snow
  • London, 1854
    • 2 million people. District of Soho, still no sewage system
    • cesspools under houses or direct to the river
    • periodic cholera outbreaks 1832, 1848, 1854, 1865, 1876
  • Good intentions, Bad decisions
    • doctors prescribing leeches, laxatives, purgatives,brandy
    • major theory behind spread of disease: “miasma” - smell.
    • At one point the London government decided to dump the waste into the River Thames. This action contaminated the water supply, leading to the cholera outbreak.
  • Stand back, I am going to try Science
    • John Snow was not convinced but the miasma theory - cholera doesn’t affect the lungs. He suspected it was something in the water but had no evidence (bacteria discovered in 1676, but germ theory only recognized in 1905).
    • Collaborated with community leader, the Reverend Henry Whitehead, to gather information from the families affected by the outbreak - went from house to house to collect data, from number of deaths to drinking habits.
    • Found that many of the people who escaped had their own wells (prison) or didn’t drink water at all (brewery).
    • Plotting deaths on a map, figured out that the Broad Street pump was at the epicenter of the 1854 outbreak (which ended up killing 616 people).

Setup: Lessons from Epidemiology

  • Map of old London
  • Hospital icon
  • People in, people out
    • other hospitals
    • back home, still sick
    • hospice
    • never get to hospital in the first place

BIRD Quality Portal

  • Boost by collaboration with Teresia Arthur (data prep) and Sumeyye Kazgan (data visualization)

WWJSD?

  • Diagnostic (buildver, sitar)
  • Prevention (CI Wall, Coverage)
  • Monitoring (Trends)
  • Containment (Drift)
  • Cure (dev teams)
  • Main page
  • BuildVer - checking the temperature
  • Sitar - CAT scan - Situation Awareness
    • concepts: impact, timeliness, drift, leaklessness
    • filtering, exploration
    • visualization alternative
  • challenges in defining concepts, collecting data
    • lack of standards to capture drift (defect movement)
    • lack of standards to define DEFECT products, components
    • lack of metadata about tracks (ship event, active, etc)
    • lack of metadata about products (source code, tests, devs, testers)
    • pervasive use of virtual teams
  • Trends - health indicators over time
  • CI Wall - not enough automated tests being executed to be useful
  • Coverage - no coverage data available at the moment
  • Drift - finding hubs, preventing contagion
    • Ebola epidemic, airport monitoring
    • network analysis: scale-free networks
    • software is a scale free network as well
    • use analysis to find hubs
    • monitor for health - bug in a hub spreads more quickly

Closing: Going forward
Empirical Software Engineering

Story (Health) | Empirical Software Engineering

As researchers investigate how software gets made, a new empire for empirical research opens up

“A growing number of researchers believe software engineering is now at a turning point comparable to the dawn of evidence-based medicine, when the health-care community began examining its practices and sorting out which interventions actually worked and which were just-so stories. This burgeoning field is known as empirical software engineering and as interest in it has exploded over the past decade, it has begun to borrow and adapt research techniques from fields as diverse as anthropology, psychology, industrial engineering and data mining.”

Greg Wilson, Jorge Aranda

http://www.americanscientist.org/issues/id.13845,y.2011,no.6,content.true,page.1,css.print/issue.aspx

Government | Organizational structure and Quality

The researchers found, however, that these organizational
complexity metrics actually could predict failure-proneness with surprising accuracy. In fact, the organizational metrics were far better predictors than characteristics of the source code itself that have been shown to be highly indicative of failures (e.g., lines of code, cyclomatic complexity, dependencies, code coverage by testing).

Motivation | Customer experience trumps technical excellence – Gartner BI reports

What are these MQs telling us?

  • Stating the bleeding obvious – technical excellence is not the same as user experience. That was OK when IT ruled the world of enterprise applications buying but doesn’t work so well when the balance of power shifts to line of business. While some of us might scoff at the notion of the CMO taking power away from the CIO, that trend is real.
  • It is a truism that technical excellence doesn’t equate to user adoption. Anyone remember the old VHS v Betamax wars? Several years ago, I saw an elegant solution in SAP Streamwork that got killed last year in favor of Jam. Why? Streamwork was a solution looking for a problem. Jam solves the problem.
  • Technical excellence doesn’t equate to customer value. Value is only achieved when software becomes pervasive. No-one questions the ROI of email because we all use it. Can we say the same about business applications that are supposed to provide real help in a data driven world? Buyers are increasingly questioning the extent to which they are paying for shelfware that was originally purchased in anticipation of end user adoption. Vendors may not be aware but a good part of the reason that shelfware exists comes from the passive/aggressive user who, in BI, would rather default to a spreadsheet than wrestle with complex BI solutions.

“SAS customers consider its software among the most difficult to use and most difficult to implement, with ease of use for business users being identified as a limitation on broader deployment by a higher percentage of customers than for any other vendor. SAS customers also report one of the highest average number of days required to create a report. Additionally, their reported three-year BI platform ownership costs were among the highest of any vendor in the survey. Furthermore, customers scored SAS’s product quality and support below the survey average. Replacing SAS’s multiple power-user-oriented tools with Visual Analytics should help to improve customers’ perception of SAS’s ease of use, but improving product quality and support must become SAS’s imperative.”

Story (Health) | Evidence-based Medicine

“the use of mathematical estimates of the risk of benefit and harm, derived from high-quality research on population samples, to inform clinical decision-making in the diagnosis, investigation or management of individual patients.”

Story (Health) | John Snow

Snow was a sceptic of the then-dominant miasma theory that stated that diseases such as cholera and bubonic plague were caused by pollution or a noxious form of “bad air”. The germ theory of disease had not yet been developed, so Snow did not understand the mechanism by which the disease was transmitted. His observation of the evidence led him to discount the theory of foul air. He first publicised his theory in an 1849 essay, On the Mode of Communication of Cholera, followed by a more detailed treatise in 1855 incorporating the results of his investigation of the role of the water supply in the Soho epidemic of 1854.[8]

By talking to local residents (with the help of Reverend Henry Whitehead), he identified the source of the outbreak as the public water pump on Broad Street (now Broadwick Street). Although Snow’s chemical and microscope examination of a water sample from the Broad Street pump did not conclusively prove its danger, his studies of the pattern of the disease were convincing enough to persuade the local council to disable the well pump by removing its handle.

====

“On proceeding to the spot, I found that nearly all the deaths had taken place within a short distance of the [Broad Street] pump. There were only ten deaths in houses situated decidedly nearer to another street-pump. In five of these cases the families of the deceased persons informed me that they always sent to the pump in Broad Street, as they preferred the water to that of the pumps which were nearer. In three other cases, the deceased were children who went to school near the pump in Broad Street…

With regard to the deaths occurring in the locality belonging to the pump, there were 61 instances in which I was informed that the deceased persons used to drink the pump water from Broad Street, either constantly or occasionally…

The result of the inquiry, then, is, that there has been no particular outbreak or prevalence of cholera in this part of London except among the persons who were in the habit of drinking the water of the above-mentioned pump well.

I had an interview with the Board of Guardians of St James’s parish, on the evening of the 7th inst [7 September], and represented the above circumstances to them. In consequence of what I said, the handle of the pump was removed on the following day.”
—John Snow, letter to the editor of the Medical Times and Gazette

====

After the cholera epidemic had subsided, government officials replaced the Broad Street Pump Handle. They had responded only to the urgent threat posed to the population, and afterward they rejected Snow’s theory. To accept his proposal would have meant indirectly accepting the oral-fecal method transmission of disease, which was too unpleasant for most of the public to contemplate.[9]

Contagion (networks) | Global Transport Networks and Infectious Disease Spread

“Air, sea and land transport networks continue to expand in reach, speed of travel and volume of passengers and goods carried. Pathogens and their vectors can now move further, faster and in greater numbers than ever before.”
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3145127/

Contagion | Software systems as complex networks: Structure, function, and evolvability of software collaboration graphs

C. R. Myers
“Software systems emerge from mere keystrokes to form intricate functional networks connecting many collaborating modules, objects, classes, methods, and subroutines. Building on recent advances in the study of complex networks, I have examined software collaboration graphs contained within several open-source software systems, and have found them to reveal scale-free, small-world networks similar to those identified in other technological, sociological, and biological systems. I present several measures of these network topologies, and discuss their relationship to software engineering practices. I also present a simple model of software system evolution based on refactoring processes which captures some of the salient features of the observed systems. Some implications of object-oriented design for questions about network robustness, evolvability, degeneracy, and organization are discussed in the wake of these findings.”

Tomcat map

Contagion (Heartbleed)

Question: How many of you changed your passwords in the past few days?

Contagion | EBOLA ALERT | Airport health teams tracking passengers from West Africa, or with fever

“A quarantine officer mans a thermal scanner to screen incoming international passengers with fever, as a precaution against Ebola virus, which has killed 80 people in Guinea, West Africa. “ [image]

Prevention | How Vaccine Fears Fueled The Resurgence Of Preventable Diseases

How lack of vaccination (fear, access) allows diseases to come back

Government (Pollution) | Duke Researchers Find Methane in Drinking Water near Fracking Sites

In a study to be released this week, researchers from Duke’s Nicholas School of the Environment found gases related to fracking in drinking water wells near drilling sites in Pennsylvania.

Done

Quotes on technology changes

AWT -> Swing

18FEB2002:
Moving to 3.0 to make sure new swing components address this issue. We won’t be fixing this in the old awt components.

Swing -> RCP

31Jul2007:
BTW, I know that … does have plans in the future to change this dialog but that probably won’t happen til RCP/Eclipse.
19May2010:
filter dialog in swing is not going to be changed … I really don’t see
anything more we could do….marking nofix

JSP -> Flash/Flex

11Mar2009:
A decision has been made that CDD resources are to be focused on Flash
technology moving forward. As products/solutions migrate to Flash technology, both JSP and SWING common components will be used less frequently. Thus, NOFIXING this defect.

If an external customer reports this as a problem, it should be opened as a CRP and will be re-evaluated. If an internal product/solution feels the issue is impacting their deliverable, they should re-open and submit it as part of the
change management process for it to be evaluated.

Flash/Flex -> HTML5

22Apr2013:
Raise the priority to high due to it’s a common flex defect that will impact
lots of products. It still reproduces on latest daily build.
21Jun2013:
Due to the beginning of the HTML Commons effort, there are fewer resources to work on Flex issues. We are deferring this defect in favor of working on other higher priority issues. If this issue becomes a customer reported problem or if it is considered critical for a solution to have fixed, please reopen with customer/solution timeline information so we can prioritize accordingly.

NOFIX for BIRD apps

NotNow

Buildver for 7.1

Get data from Buildver (from 6.4 as mockup)

  • Table is in \bqp\buildver\6.4\table.html, but would need to expand to create line charts in VA. Might be better to just hack the PHP script to generate CSV instead (and port it to Ruby in the process)

Create Designer report with data from Buildver

ToDo