To the front pageThe Interaction Designer's Coffee Break - Weekly postings and quarterly articles about interaction design  
  To the front pageSign inTo the frontpageSearch in GUUUI postingsAbout GUUUI  
   
 

ISSUE 03 - Q3 2002

Results from a survey of web prototyping tools usage

In June 2002 GUUUI conducted a survey on web prototyping tools usage. The purpose of the survey was to find out what tools are used for prototyping, what requirements interaction designers have for their tools, and how happy they are with the tools they are using. This article presents the results of this survey.

Some key findings:

  • There is currently no distinct consensus on what is the best tool to use for web prototyping
  • HTML and diagramming tools are the most commonly type of tools used for web prototyping
  • Macromedia Dreamveawer and Microsoft Visio are the most widely used tools
  • Respondents feel that the tools they use fall short on some high priority criteria, such as laying out and making changes to pages
  • Few seem to be perfectly happy with the tools they are using though users of HTML tools are strikingly more content than users of diagramming tools

What tools are used for prototyping?

The survey confirmed what discussions on mailing lists such as CHI-WEB and SIG-IA have indicated: there is currently no agreement on what is the best type of tool to use for web prototyping. However, there is a majority of 28.3% using a visual or text based HTML tool as their primarily type of prototyping tool, closely followed by 24.5% who prefer a diagramming tool.

 
     
  "What is the primarily type of prototyping tool you use for prototyping?" (52 replies)

 
  Non-computerized tool such as pen and paper (paper prototyping), white boards, Post-It notes and overheads 7.5%  
  Presentation software such as PowerPoint from Microsoft 9.4%  
  Graphic design tool such as Illustrator or Photoshop from Adobe 18.9%  
  Diagramming tool such as Visio or SmartDraw 24.5%  
  Visual or text based HTML tool such as FrontPage, Dreamveawer or HomeSite 28.3%  
  A software development tool such as Microsoft Visual Basic or Borland Delphi 1.9%  
  A multimedia tool such as Director from Macromedia or Authorware from Macromedia 0.0%  
  Other 9.4%  
     

Responses to the question "What is the name of the prototyping tool you are using" showed that among the respondents using HTML tools, Dreamweaver is most widely used, while almost all respondents using a diagramming tool depend on Visio.

How happy are designers with the tools they are using?

Asking how happy interaction designers are with the tools they are using reveals that there is few shouts joy. While designers using a HTML tool tend to be pretty content, respondents using a diagramming tool are surprisingly dissatisfied.

   
     
  "I'm perfectly happy with the prototyping tool I'm using"
    Strongly disagree Mildly disagree Neither agree nor disagree Mildly agree Strongly agree
  All replies (52 replies) 7.5% 37.7% 15.1% 32.1% 7.5%
  Using a visual or text based HTML tool (15 replies) 6.7% 20.0% 6.7% 60.0% 6.7%
  Using a diagramming tool (13 replies) 7.7% 76.9% 7.7% 0.0% 7.7%
  Using a graphic design tool (10 replies) 10.0% 20.0% 20.0% 50.0% 0.0%
     

What are the requirements?

In the survey, the respondents where asked what they require from a web prototyping tool, and to what extend the tools they are using live up to the very same requirements. The purpose of these questions was to compare the features of the prototyping tools with the designer's requirements.

The results showed that prototyping tools primarily have to support:

  • Quick and easy revision of pages
  • Quick and easy layout of pages
  • An easy way to demonstrate the prototype to clients and team members
  • Quick and easy site wide revisions

Less important is it that a web prototyping tool:

  • Is able to simulate advanced functionality (e.g. scrolling and selecting items from a drop-down list)
  • Is easy to learn
  • Produces good-looking prototypes

The answers about whether the tools that the respondents use live up to their requirements showed that the tools fall short on some high priority criteria. They primarily fail in supporting:

  • Quick and easy revision of pages
  • Quick and easy site wide revisions
  • Quick and easy layout of pages
   
     
  This table compares the questions "Considering your requirements for a web prototyping tool, to what extend do you agree with the following statements?" (first row against the question) and "Considering your personal requirements for a web prototyping tool, to what extend does the tool you use live up to the following statements?" (second row) (52 replies)
    Strongly disagree Mildly disagree Neither agree nor disagree Mildly agree Strongly agree
  Quick and easy to layout pages 0.0% 0.0% 3.8% 15.1% 81.1%
  2.0% 9.8% 7.8% 39.2% 41.2%
  Quick and easy to make changes to pages 0.0% 0.0% 0.0% 11.3% 88.7%
  3.9% 19.6% 15.7% 35.3% 25.5%
  Quick and easy to do site wide revisions to the prototype 0.0% 0.0% 3.8% 30.2% 66.0%
  19.6% 33.3% 13.7% 13.7% 19.6%
  Able to simulate basic functionality (e.g. link between pages) 0.0% 3.8% 20.8% 35.8% 39.6%
  21.6% 7.8% 13.7% 19.6% 37.3%
  Able to simulate advanced functionality (e.g. scrolling and selecting items from a drop-down list) 1.9% 13.2% 30.2% 41.5% 13.2%
  32.7% 13.5% 7.7% 21.2% 25.0%
  Easy to demonstrate the prototype to clients and team members 0.0% 0.0% 0.0% 32.1% 67.9%
  0.0% 13.5% 11.5% 21.2% 53.8%
  Suitable for carrying out usability testing 0.0% 7.5% 20.8% 32.1% 39.6%
  5.8% 17.3% 23.1% 32.7% 21.2%
  Produce a good looking prototype 9.4% 9.4% 26.4% 39.6% 15.1%
  0.0% 13.5% 19.2% 34.6% 32.7%
  Fit in with existing development tools 3.8% 21.2% 32.7% 26.9% 15.4%
  0.0% 9.6% 46.2% 26.9% 17.3%
  Inexpensive 1.9% 7.5% 41.5% 30.2% 18.9%
  3.8% 13.5% 28.8% 28.8% 25.0%
  Easy to learn 1.9% 11.3% 28.3% 30.2% 28.3%
  5.8% 25.0% 11.5% 44.2% 13.5%
  Easy to use 0.0% 0.0% 11.3% 45.3% 43.4%
  1.9% 11.5% 21.2% 40.4% 25.0%
     

To be able to compare the different tools' support of the respondent's requirements, I've compiled a table summarizing the ratings of the individual tools. Most interestingly, the table shows that:

  • HTML tools have a high overall score, with site wide revision being the major problem
  • Diagramming doesn't score high on any criteria, and is considered particularly poor at supporting site wide revisions and simulating advanced functionality
  • The respondents using a presentation tool find that their tools are very easy to use and suitable for presentation and setting up pages
  • Respondents using non-computerized tools feel hat the approach fall short when having to do site wide revisions
   
  This table summarizes the replies to the single choice matrix question "Considering your personal requirements for a web prototyping tool, to what extend does the tool you use live up to the following statements?" (the numbers are calculated as "neither agree nor disagree" divided by two plus "Midly agree" plus "Strongly agree).
    HTML tools Diagramming tools Graphics tools

Presentation tools

Non-computerized tools

    (15 repl.) (13 repl.) (10 repl.) (5 repl.) (4 repl.)
  It's quick and easy to layout pages 90.1% 69.3% 77.8% 100.0% 87.5%
  It's quick and easy to make changes to pages 90.0% 57.8% 61.1% 80.0% 37.5%
  It's quick and easy to do site wide revisions to the prototype 73.3% 15.4% 16.7% 60.0% 25.0%
  It's able to simulate basic functionality 93.0% 53.9% 35.0% 80.0% 50.0%
  It's able to simulate advanced functionality 93.3% 15.4% 35.0% 20.0% 37.5%
  It's easy to demonstrate the prototype to clients and team members 83.4% 50.0% 95.0% 100.0% 100.0%
  It's suitable for carrying out usability testing 76.7% 53.9% 50.0% 50.0% 87.5%
  It produces good looking prototype 96.7% 57.8% 100% 60.0% 50.5%
  It fits in with existing development tools 73.3% 57.7% 85.0% 60% 50%
  It's inexpensive 60.0% 61.6% 65.0% 100.0% 87.5%
  It's easy to learn 60.1% 46.2% 60.0% 100.0% 75.0%
  It's easy to use 83.3% 50.0% 60.0% 100.0% 100.0%
     

Thanks to Jens Hørup and dubidu for sponsoring this survey.

Text: Henrik Olsen

   

 
     
  To the front pageSign inTo the frontpageSearch in GUUUI postingsAbout GUUUI