Institute for Innovation in Business and Social Research (IIBSoR)

About IIBSoR

Improving Business Performance

Improving Society and Environment

Consulting and Contract Research

Papers and Publications

Why do a PhD?

Positions Vacant

Intranet

Contact Us

News and Events

AES Tech-Eval

These days it is no surprise to see mainstream and niche programs making use of tech-based platforms: web-based self-help tools, mobile applications, SMS-based reminder systems, viral videos, conversations on social media... the list is much longer than this, and ever growing.

There is a need to develop capacity among evaluators to work confidently in this environment, designing and executing sound evaluations that understand what these technologies are, how they can be used and how their impact can be measured.

There are also great opportunities for using technology in our evaluations – wikis, online forums, online surveys, social media monitoring... again the list is long and growing.

Spilling over from one of the parallel sessions at the 2011 Australasian Evaluation Society Conference a crew of around 15 people has started pulling together a new AES Special Interest Group around this intersection between evaluation and technology: AES tech-eval.

Join the mailing list

To connect with others in the AES tech-eval group, add your email address to the mailing list.

Go on, sign up! If technology freaks you out, swap fear of the unknown with curiosity and see where it takes you. If you’re already working comfortably in this space, help lead your colleagues forward.

If you have trouble with the mailing list, contact Duncan Rintoul.

Resources and links

This site also serves as a repository of relevant published evaluations, resources and links.
If you can add resources to this list, contact Duncan Rintoul.

The Reach Out Website (Inspire Foundation) aims to promote mentally healthy behaviours among young people. It has been evaluated by Atari Metcalfe over several years.

Atari’s slides from the 2011 AES conference (pdf)

The wellbeing toolbox is a self-help website for veterans and their families. Del Lloyd (Australian Centre for Posttraumatic Health) presented an evaluation of the site at the 2011 AES conference.

Del’s slides from the 2011 AES Conference (pdf)

Bursting the bubble is a website targeting young people, designed to prevent relationship violence. It was evaluated by Brad Shrimpton and colleagues at Melbourne University Centre for Program Evaluation.

The evaluation findings are summarised online. The findings were also summarised in a guide to developing websites for young people (first half, second half).

TAFE NSW used to run an intranet-based Professional Development Network. Larraine Larri (Renshaw-Hitchen & Associates) presented a poster on its evaluation at the 2003 AES Conference (Auckland).

Key elements of this poster (pdf)

Social software includes things like wikis, web forums, online focus groups, online surveys, digital photostories and video-blogging.

Larraine Larri (Renshaw-Hitchen & Associates) has contributed to several resources on use of social software, in evaluation and otherwise.

2006 Australian Flexible Learning Framework paper co-authored by Larraine with Val Evans: Networks, connections and community: Learning with social software

Larraine’s 2007 and 2009 AES Conference roundtable session on use of social software in evaluation practice:

B.J. Fogg's work at the Stanford Persuasive Technology Lab sometimes falls outside the scope of traditional public health and social marketing, but is well worth a look.

http://www.bjfogg.com/

FuelZone aimed to increase the uptake of school meals among children in Glasgow aged 11-16.

The project used a mix of canteen redesign and a new cashless payment card system which provided users with points (that varied on the ‘healthiness’ of their food choices) that could be used to redeem prizes online (eg i-pods, cinema tickets) as an incentive.

This case study is taken from the ShowCase database of evaluated social marketing programs, curated by the (UK) National Social Marketing Centre .

FuelZone case study page on ShowCase

FuelZone full case study (pdf)

FuelZone summary (pdf)

Other case studies on ShowCase have had web based elements (eg a website), but the performance of these technologies has not often been measured all that well in the evaluation.

Three that have included some form of measure of their tech-based interventions are listed here. However, none of the evaluations are particularly precise about the tech-based aspects.

Take Charge. Take the Test. – a one-year social marketing campaign in Cleveland (OH) and Philadelphia (PA) to increase HIV testing in African American women at high risk for HIV infection. The www.hivtest.org website showed people where they could find out where they could find the closest testing facility was.

Fight Back aimed to reduce smoking prevalence in male routine and manual workers aged 35 to 55 living in a deprived part of the UK. The campaign included this viral video.

Be a star aimed to promote breastfeeding amongst 16- to 25-year-old mothers in central Lancashire. It included a two-way SMS system to provide 24-hour advice and support to mothers via text in the 12 weeks following hospital discharge, as well as a website and blog to provide support, information and signposting for women, partners and families on a range of breastfeeding and wider issues.

There is a tremendous amount of material about energy efficiency and the smart grid starting to appear in Europe, North America, and other parts of the world.

In addition to the links here, other potential avenues include: