Plat_Forms 2007 evaluation data:
Explanation of the data files and variables therein
=====================================================
created: Lutz Prechelt, 2007-02-07
see http://www.plat-forms.org

Section 1 describes variables that occur in several different files.
Section 2 describes the data files (or sets of data files) and the specific
variables that occur in each.
Many files are members of groups of related files. A part of the name in
parentheses means that this part is variable. For example a name such as
  completeness-(team)-initial-(reviewername).csv 
represents a group of files such as
  completeness-team1-initial-ertel.csv
  completeness-team2-initial-ertel.csv
  completeness-team1-initial-thiel.csv
etc. and all of these files share the same structure and variables.

All files containing structured data have tab-separated-value format
and use the .csv suffix. Most of them were created by OpenOffice Calc.
A few files have other types.

The files are stored in different subdirectories depending on their main
author. These directories are not mentioned here.


Section 1: Generic variable names
----------------------------------

(in alphabetical order)

date
  The date part of a timestamp, in ISO format yyyy-mm-dd

req
  A requirement number according to the document "Plat_Forms 2007 task: PbT"

team
  Team number, with the following meaning:
    1	Perl  Etat de Genève/Optaros
    2	Perl  plusW
    3	Java  abaXX Technology
    4	Java  Accenture Technology Solutions
    5	Perl  Revolution Systems
    6	PHP	  OXID eSales
    7	PHP	  Globalpark
    8	PHP	  ZEND Technologies
    9	Java  Innoopract Informationssysteme
            
time        
  The time-of-day part of a timestamp in format hh:mm or hh:mm:ss
            
            
            
Section 2: Description of files and variables
---------------------------------------------

(in alphabetical order)

browsertroubles.csv
  Each entry is one aspects where behavior or (more typically) presentation
  was different from one browser to another in an obviously unintended way
  for one particular pair of solution (team) and browser.
  We separately list each browser in which this looks "wrong".
  The list is by no means complete. It just contains what we found during
  the systematic testing using specific scenarios (as described in the
  evaluation report) and what we stumbled across during other use of the
  applications (e.g. during completeness testing).
  Variables:
    team
    req
    reviewer
      Name of the person who has created this entry.
      One of ertel, hardy, prechelt, thiel.
    browser
      The name of the affected browser. Value must be one of the following:
      Firefox, IE, Opera, Safari, Lynx.
      The browser versions we used were
      Firefox 2.0, IE 7, Opera 8/9, Safari ???, Lynx ???
    description
      a short verbal description of what goes or looks wrong

completeness-(team)-initial-(reviewername).csv
  team is team1...team9.
  reviewername is ertel or hardy or thiel and represents the file's main author
  Each entry in such a file grades the completeness of implementation for one
  requirement in one solution.
  Variables:
    team
    req
    completeness
      An integer 0...4 indicating how completely the requirement was
      implemented by this team. Meaning of the values:
        0: missing
        1: only partially implemented
        2: fully implemented, but in a particularly bad way
        3: fully implemented
        4: fully implemented, but in an extraordinarily good way
      0 and 3 should be the standard values,
      1, 2, and 4 require and explanation
    explanation
      Usually empty for completeness 0 and 3.
      For completeness 1, 2, or 4, describes in short text form what is
      incomplete or what is so bad or so good about the given implementation.

defects.csv
  Each entry is one aspect of one solution that behaves clearly different
  from the specification given in the requirements document or obviously
  different from what the solution authors have intended.
  The list is by no means complete. It just contains what we found during
  the systematic testing using specific scenarios (as described in the
  evaluation report) and what we stumbled across during other use of the
  applications (e.g. during completeness testing).
  Variables:
    team
    req
    reviewer
      Name of the person who has created this entry.
      One of ertel, hardy, prechelt, thiel.
    description
      a short verbal description of what is wrong

deliverables-full.csv
   Describes the deliverable file submitted by each team at the end of the
   contest and the three files contained in it. See the requirements
   document section 5 for details.
   Variables:
     team
     delivDate
     delivTime
       When the deliverable (or more precisely: The email containing its
       SHA-1 checksum) was submitted.
     delivSize
       Size of the deliverable file in MB
     delivType
       Type of the deliverable file
     vmSize
       Size of the VMware virtual machine archive file in MB
     vmType
       Type of the VMware virtual machine archive file
     srcSize
       Size of the source code distribution file in MB
     srcType
       Type of the source code distribution file in MB
     verSize
       Size of the source code version archive archive file in MB
     verType
       Type of the source code version archive archive file
     verSW
       Type of the versioning software used. One of svn, cvs, perforce.
     delivSha1Email
       SHA-1 checksum sent in the submission email
     delivSha1Actual
       SHA-1 checksum of the actually delivered file
     vmSha1
       SHA-1 checksum of the VMware virtual machine archive file in MB
     srcSha1
       SHA-1 checksum of the source code distribution file
     verSha1
       SHA-1 checksum of the source code version archive archive file

fileversions-teamX.csv
  There is one such file for each team's solution, named
  fileversions-team1.csv etc.
  Each record represents a checkin record for one version of one file.
  Variables
    date
    time
    author
      Username of the person performing the checkin
    filename
      Filename, including relative path
    version
      Version number of this file (for CVS) or of the repository (for SVN).
    filetype
      The suffix of the filename, such as java, php, txt, xml etc.
      Is empty for filenames without suffix.
    size
      Total size in bytes
    lines
      Total size in lines
    linesadded
      Number of lines added relative to previous version according to 'diff'.
      Is equal to lines for version 1.
    linesdeleted
      Number of lines removed relative to previous version according to 'diff'.
      Is 0 for version 1.

postmortem-answers.csv
  Record of the answers that each particpant gave in the postmortem
  questionnaire. See postmortem-encoding.rtf for the coding scheme.
  The less structured answers are only sized here, not actually recorded.
  Variables (see postmortem-encoding.rtf for description):
    team
    organization
    age
    experience
    education
    devtimePercentage
    estCapabilityPercentile
    languagesUsed
    languagesUsedList
    languagesTried
    languagesTriedList
    frameworksTried
    frameworksTriedList
    estEffortWebdesign
    estEffortProgramming
    estEffortProgrammingMay
    estEffortQA
    difficultAspects
    advantageAspects
    weakAspects
    filesToRead
    filesToReadKnownDegree
    filesToReadNewbie

process-observations.xls
process-observations.csv
  process-observations.csv was created from the .xls by
  - saving the .xls in MS Excel as tab-separated .txt,
  - renaming it to .csv, and
  - manually removing the non-data lines at the top and bottom.
  The file contains the observations we recorded on paper during the contest
  every 15 minutes: one activity category status marker for each team member.
  The following activity category markers are used:
	T	appears to be thinking (or scribbling on paper etc.)	 
	RS	reads the specification (task description, on paper or as PDF)
	D	uses development environment (IDE, Editor, shell, graphics program etc.)
	D2	uses development environment together with a 2nd team colleague (use D3 if needed)
	E	uses email client
	BP	uses a web browser, presumably on the prototype
	BO	uses a web browser on some other page
	CO	uses the computer HW/SW in some other way
	TT	talks to someone from same team
	TO	talks to someone other (e.g. the customer)
	PF	pauses to consume food or drink
	PP	pauses in some other way, but is present
	PA	pauses and is absent (not in the room, but check neighboring rooms)
  Variables:
    date
    time
    tm11
      activity category of team 1, team member 1
    tm12
      activity category of team 1, team member 2
    tm13
      activity category of team 1, team member 3
    tm21
      activity category of team 2, team member 1
    [...and so on, until tm93]
    who
      code of the person doing the categorization in this timestep:
      F: Florian Thiel, P: Peter Ertel, W: Will Hardy, Z: several together

questionslog.csv
  Each entry represents one question that one team asked the customer during
  the contest.
  Variables:
    date
    time
    team
    req
      Which requirement the question refers to. (Is often not applicable.)
    questionType
      One of the following
        clarification: request for clarification of a requirement
        error: inconsistencies in the requirements document
        extension: possible requirements beyond those given in the req document
        misunderstanding: a question based on a misunderstanding of some req
        procedure: how to behave as a contest participant
    questionTopic
      a short paraphrase of the question. For a longer one see questionlog.txt

questionslog.txt
  The long textual form of the questions described in questionlog.csv.
  This file was created manually by the customer during the contest.

req-types.csv
  Classification of the requirements. Most of this is taken directly from the
  requirements document.
  Variables:
    reqPriority
      One of MUST, SHOULD, MAY
    req
    reqCategory
      One of UI (for reqs from section 2), Webservice (section 3),
      Nonfunctional (section 4), Rules (section 5)
    reqArea
      One of registration (section 2.2), ttt (2.3), search (2.4),
      memberlist (2.5), statuspage (2.6), webservice (3),
      ui (4.1), browser (4.2), scalability (4.3), persistence (4.4),
      timeout (4.5), style (4.7), blog (5.3), delivery (5.5)

sleeping.csv
  A record of the time when the whole team was absent from the contest room
  during the night/morning of 2006-01-26. 
  Note that some teams sorf of took turns working, so there was sometimes
  more sleep than you can see here.
  Variables:
    team
    absenceBegin
    absenceEnd

teamestimates.csv
  There were two quick informal surveys held among the teams during the
  contest:
  - At about 12:30, 3.5 hours into the contest, the teams were asked
  to estimate when they would put their first preview prototype online. One
  team answered "probably never", recorded as inf (infinity), the others gave
  some earliest plausible time.
  - At about 10:00 on the second day, 25 hours into the contest, the teams
  were asked what fraction of the MUST/SHOULD/MAY requirements they expected
  to implement by the end of the contest (in percent).
  Variables:
    team
    estPreview
      expected time when first prototype would go online
    estMUST
      expected percentage of MUST requirements that would be implemented
    estSHOULD
      expected percentage of SHOULD requirements that would be implemented
    estMAY
      expected percentage of MAY requirements that would be implemented
