Campaign Metrics

From UrbanWiki

Jump to: navigation, search

This project focuses on standardizing the logging of typical measurements for performance, usability, and user participation (reputation) to a common format/access to a centralized server. For ease of access, aggregation, and propagation.

Contents

[edit] Specification

For more information: Campaign_Metrics_Specification

[edit] Metrics

The following are proposed metrics that the system should support. Please add/remove as necessary. Calculations are accepted, but measurements are preferred. For example, measurement: timestamp of first data point; calculation: upload frequency for campaign.

[edit] Participant

  • # of campaigns participating in 
 measure(['participant', guid('uid.example@domain.tld'), 'join'])
  • # of campaigns abandoned 
 measure(['participant', guid('uid.example@domain.tld'), 'part'])
  • Upload frequency 
 measure(['participant', guid('uid.example@domain.tld'), 'post'], date(), 'date')
  • Use of Beta distribution for confidence/reputation 
 #todo  helper.beta(namespace, namespace) # returns: mean, confidence

[edit] Contributions

  • Count of "invalid," and other tags/attributes, of images 
 measure(['participant', guid('uid.example@domain.tld'), 'data.invalid'])
  • "Quality," using common rating convention among the campaign (ie. 5 stars), of images 
measure(['participant', guid('uid.example@domain.tld'), 'data.quality'], 3)
# or
measure(['data', guid('uid.picture.png'), 'quality'], 4)

[edit] Campaign

  • # of participants 
 # Accessible from participant namespace
  • lifetime 
 # Accessible from the Date statistics using the first and last column
  • data coverage (area, period) 
 # Accessible from the Location statistics using the area column

[edit] Access

  • Frequency a participant checks in with the system 
 # code
 measure(['participant', guid('uid.example@domain.tld'), 'visit'], date(), 'date')

 # Accessible via participant.GUID_of_participant.visit.statistics.frequency
  • Frequency/count of upload method use 
 # code
 measure(['participant', guid('uid.example@domain.tld'), 'data.upload.method'], 'email', 'string')

 # Accessible via participant.GUID.data.upload.method.statistics.values
    • web form
    • email
    • sms
    • client software (ie: campaignr)
  • Upload delay (server receiving timestamp - capture timestamp) 
 # code
 measure.start('data.upload.delay')
  ...
 measure.stop('data.upload.delay')

 # Accessible via data.upload.delay.duration.mean
  • Network technology use for upload 
 measure('data.network', 'gsm', 'string')
    • GSM
    • 3G
    • WiFi
  • Device used (using HTTP_AGENT) 
 measure('device', 'iphone', 'string')

[edit] Performance

  • Latency between data processing modules
  • Crashes - log when app crashes whenever possible.
  • Battery life, signal strength, etc. w/ info about what sensors are in use.

[edit] Interaction

  • For applications that have multiple "Activities"/views/functions log frequency of usage of each. Eg. How often do participants visit the stats or summary screen.
  • How often, and delay, does a participant respond to triggers?
  • How often do they launch the application on their own without any prompt or reminder.
  • Log turning on and off sensors and services by participant.

[edit] Privacy

  • For applications that have privacy-preserving features, log frequency of usage of each. For example:
  • How frequently does a user delete data?
  • How frequently does a user replace data or routes with new data or routes?
  • If the default is sharing data, how often does the user opt to not share data?
  • If the default is not sharing data, how often does the user opt to share data?
  • How many users have deleted accounts?

[edit] References

[1] S. Reddy, K. Shilton, J. Burke, D. Estrin, M. Hansen, and M. Srivastava, “Evaluating participation and performance in participatory sensing,” in Proceedings of the International Workshop on Urban, Community, and Social Applications of Networked Sensing Systems (UrbanSense08), Nov. 2008.

Retrieved from "http://wiki.urban.cens.ucla.edu/index.php/Campaign_Metrics"