Website Survey - Health Research for Action Analysis of ARB's Website

This page last reviewed April 14, 2010

add description

The 2007 public survey results have provided the path for the California Air Resources Board Information System (CARBIS) Steering Committee initiatives (see the Executive Summary - a Living Document).  An unsolicited analysis of our website was presented to us January 2008 by the Health Research for Action (HRA).  Their analysis totally validates the results of the larger 2007 survey.  The following provides the HRA findings and the CARBIS Committee's completed responses to this analysis including future action items indicated in blue text.  Clearly, we have not resolved all our issues, but a lot has changed and we are working continually on these mostly valid criticisms.


1.    Goals and Target Audiences Not Well Defined

A successful website balances an organization's goals with the needs and abilitites of its target audiences.  A site based on this strategy intuitively directs its audiences down paths that ultimately (and quickly) allow both the audiences and the organization to achieve their goals.

Although ARB may have very well-defined goals and target audiences, these are not clearly apparent on its website.  The following contribute to the site's lack of clarity:

  • Ineffective Communication of Identity
    The ARB Website does not effectively communicate its own mission and goals and how it serves Californians.  This is apparent on the homepage and throughout the website.

    ARB's mission statement and goals are located under the link "About the ARB" which in turn is linked from the homepage.  ARB's Stretegic Plan needs updating.  

  • Poorly Defined Target Audiences
    The ARB website does not clearly identify its target audiences.  Furthermore, it does not provide logical pathways for users to follow based on their needs.  This has to negative results:
    • It is difficlt for users (e.g., California's diverse residents; scientists; businesses; local, regional and federal policymakers and analysts; advocates, educators; journalists) to find the information they need.
    • ARB is unable to effectively deliver relevant messages to the appropriate andiences.

      1.  In targetting ARB's many audiences, the HRA analysis did not once mention ARB's list serve / RSS features.  This is frustrating since this is the major way in which we connect with our individual  stakeholders.  So, how granular is this process?  We host approximately 200 subscriber lists, plus over 275 "comment" lists developed for each of our formal docket sessions.  These lists include over 400,000 subscriptions (over 140,000 unique email addresses).  In 2008, we sent out over five million emails to these various stakeholder groups.  All list serve broadcasts are archived by list topic and also chronologically in our What's New / RSS database.  For stakeholders unwilling to provide us an email address, all this information is available via RSS news feed.  The website of over a quarter million documents may be the archive of ARB activity, but the way in which we attempt to engage our individual stakeholders is via list serve email.  This is THE way in which most stakeholders learn of updates to their areas of interest on the site.  Is there any equally assertive email campaign in California State service? 

      2.  ARB has implemented a "User Portal" tab feature linked from ARB's homepage.  We need to flesh this out further.

Because of its pioneering role in protecting public health and the environment, ARB has the potential to lead the state and the nation in addressing the health effects of climate change in addition to its mandate to reduce air pollution.  ARB's present website does not adequately reflect its mission or potential, particularly in ways that allow the typical user to understand and take action related to the impacts of climate change on health.

The climate change area of our website is certainly our fastest growing area since January 2008.  Since then, web content has grown more than ten fold and we broadcast to no less than 40 climate change related list serve stakeholder groups.  The climate change area has its own meetings calendar and we have established multiple inter-departmental outreach efforts.  The website design borrows heavily from the State of California portal which has won some impressive awards recently.  Not surprisingly, ARB's climate change website today bears little resemblence to that of January 2008.

2.    Poor Usability

The ARB website is generally difficult to use.  While business users may have taught themselves to find relevant information quickly because it is essential to their businesses, new and general users must work hard to find what they need.  Moreover, although more thatn 39% of California's population speaks a language other that English at home, ARB's website is not navigable in Spanish, only selected materials are available in Spanish, and no content is available in languages other that English or Spanish.

ARB does not have a viable Spanish web presence, and we have made virtually zero progress on this in the past year.

The difficulty with using the ARB website is the combined result of goal and targetting issues (mentioned above), and the following problems related to information architecture and tactical navigation, which require extensive scrolling to find desired einformation.
  • Information Is Not Categorized or Presented by Audience Need
    Generally, information on the ARB website is not categorized by its relevance to user need -- which requires users to investigate several paths to find relevant material.  This is an inefficient way to gather data.

    For example, the nested tab display on the home page is divided into three sections:  "Resources," "Health," and "Education."  These categories are not particular to any one user type, and the information may or may not be relevant to all users.  Users must therefore dig through the categories and follow unspecific links to seek relevant information.  There is no way for users to know if the "Fact Sheets" contain personally relevant items unless they click the link and sift through a very long list spanning many different subjects.

    This is true.  The CARBIS Committee has not been able to group its stakeholders in three or four categories.  Instead, we have implemented a fourth tab on our homepage called "User Portals" with the intention of populating this with stakeholder portals to provide an additional path to the materials.  With the consumer vs. operator vs. manufacturer portal pages for mobile sources linked from this tab, we have made some headway in this effort.  What are the other stakeholder groupings that make sense given that ARB touches on every citizen in the State but in so many ways?

  • No Hierarchy of Information

    In many sections of the ARB website, it seems as though information has been randomly placed on the screen.  The result is a frustrating experience for the user.  A clear hierarchy of information would make it much easier to find and review relevant information.

    In 2008, ARB implemented a new customized left hand nav bar structured into two parts: "UP LINKS" and "LOCAL LINKS."  Given the web-like reality of ARB's program structure, the "up links" feature enables mulitple "bread crumb" possibilities.  If a user finds herself on the Carl Moyer (diesel incentives) Program top page, and finds that this is too much detail, the up links provide a path up to no less than three branches "mobile sources," "diesel" of other "financial incentive" programs. The bottom bolded up link identifies the family of pages the user is currently navigating.  The "local links" area of the left-hand nav bar provides an alphabetized listing of document types found within the family.  In most cases, we are using a consistent naming convention across the entire site for these local links.  We have added customized left hand nav bars to over 4,500 top level webpages with about 2,000 pages yet to be converted.

  • Few Primary Calls-to-Action
    On many pages, including the home page, the primary call-to-action is unclear.  Users are unsure about why, how, and where to proceed.  Literature on environmental communication has shown that the lay public and scientists have very different attitudes towards risk, and information needs to be targeted in ways that help the public understand and act on information about environmental hazards.

    We have not adequately addressed this issue.  Hopefully, our general public visitors are starting to click on our new "Learn more...." homepage feature which provides a link to the "50 Things" you can do to help clean the air in California.  Additional "tips" and "facts" will be added to this new javascript carousel of images and associated narratives.  It is worth considering changing the name of this feature to "Call to Action" and make these inspirational narratives linkable from both the homepage and from their respective website topic area.  We should consider a new document type for inclusion in the customized left hand nav bars called "Call to Action."  Also, we are trying to get ARB staffers to include clearer "call-to-action" commentary in our background documents which are intended to be 8th-grade English readable.

  • Inconsistent Navigation
    Throughout the site, the 2nd, 3rd and deeper level navigation varies greatly from page to page.  The site's lack of a standard navigational model is confusing to users, who must continually learn the new navigation schemes.  Furthermore, inconsistent navigation makes it difficult for users to maintain a sense of place while navigating through the site.  This makes it very easy for user to get lost while on the ARB website.

    With the implementation of our customized left hand nav bars, some of this problem is now gone.  As these consistently-named "local links" are moved to the left column, webpages are getting shorter.  The 2007 public survey made it clear we are too wordy, and hence our push to come up with more top level portal pages with all the links, but with little or no explanatory narrative.  Let the user click, and on the resulting page, provide the explanation.  We suspect this is particularly helpful for our power users; i.e., the 60% of our 2007 survey participants who visit the site at least twice per week.

  • Inconsistent Design and Functionality
    The design differences from page to page are dramatic.  Users definitely prefer a consistent model for look-and-feel and functionality.  Like inconsistent navigation, inconsistent design disrupts a sense of place and makes the website difficult to use.

    New xhtml 1.0 strict compliant templates have been built and are being implemented as time permits.  This page is built in this new template.  All one- and three-column webpages are being converted to two-columns in the new template.  Fonts will become more consistent as we migrate to the new template.  Note that this template conversion is something we have to do to bring the website into ADA compliance.  Removing all "cartoonish clip art" from the site is complete, and adding an photo image to all top level webpages will add visual interest to the site as well.

    Note of inconsistent navigation, design, and functionality:  Dramatic inconsistencies in navigation, design, and functionality can have more severe consequences than increased user burden and/or loss of place.  Such inconsistencies can adversely affect users' perception of the entire organization.  A cohesive, clean, efficient, and usable website will build user confidence in the organization, while the opposite may destroy confidence and lead users to question the professionalism and capability of the organization.