Remapping LA

From UrbanWiki

Jump to: navigation, search
Nokia N810 tablet
Nokia N810 tablet

REMAP is undertaking a long-term research effort to engage urban communities in the design of technological systems that express their cultures and identities. The multi-year project, titled REMAPPING LA, features a decentralized approach to technological development, which is centered on three key principles: 1) Participation of the general populace, with an emphasis on the involvement of Los Angeles teenagers, families, and educators; 2) Awareness of the context in which the systems are being designed; 3) Encouragement of tangible and meaningful interfaces between project members and media.

Official website: [1]

Project tracking website: [2]

Contents

[edit] Nokia N810 system

The Nokia N810 software targets to provide access to a rich and event-triggered media interface for touring and authoring tours in the urban landscape.

The software is written in Adobe Flash 9 (ActionScript v3) and Python and can be run on N810 tablets with or without network access.

[edit] Features

  • Map based virtual tours
  • Custom map overlays
  • Rich media support (text, images, sound, and images)
  • Location aware (GPS)
  • Event-triggered
  • Argumented reality overlay (based on AR toolkit)

[edit] Schedule (spring, 2009)

  1. Stabilize/enhance
  2. Implement slideshow module
  3. Implement Authoring tool
  4. Experiment w/ Argumented Reality (AR) toolkit (ATR)

Week 5 (26-1)

  • Refactor Action Script code for optimization purposes (makes sure GC is run often enough, resources are released, etc)
  • Start implementing slideshow module
  • Access size of map tiles in memory

Week 6 (2-8)

  • Plan how-to move modest maps from web-service to file-based service
  • Continue optimization work
  • Enable kiosk mode (browser can not be killed)

Week 7 (9-15)

  • Finish first prototype of slide show module
  • Continue optimization work

Week 8 (16-22)

  • Evaluate work (slideshow prototype, optimization gain, use cases scenarios)
  • Create documentation
  • Determine hardware requirements of AR

Week 9 (23-1)

  • Outline Authoring feature specs
  • Start implementing authoring prototype
  • Determine performance experiences of ART on embedded platforms+

+ N810 Hardware configuration of N810: TI OMAP 2420, 400 Mhz, 128 MB RAM

Week 10 (2-8)

  • Continue work on Authoring features (revise feature specs)
  • Determine portability of ART to OMAP2/debian
  • Cross-compile and run AR toolkit sample application on the N810

Week 11 (9-15)

  • Continue work on implementing Authoring features
  • Developer AR prototype on N810, if considered feasible
  • Wrap up all work and create documentation

[edit] Developer Documentation

[edit] The MediaViewer

The viewer has two modes of operations: grid and slides.

In the grid view one can show a variable or a fixed number of media thumbnails (including sounds files). Each file has a "show on map" button and the user can show the media file w/ meta-information in full screen my tapping the thumbnail. The view can dynamically adjust in size to the actual number of items or enforce a predefined number of fields.

In the slides view one foreground media is show at a time. The duration can be specified by the user or set according to the inherent duration of the media (the later applies to sound and video only). A background track containing sound or text can be played in parallel to the foreground media. Sounds will play until the slide show ends or is terminated by the user. Text will be shown next to the media currently in the foreground.

Accepted data format (class: QueueItem): 

 {id,rank,title,type,url,group,duration,tooltip,location}

[edit] Clustering of media

Each thumbnail placeholder has an independent media queue associated; the thumbnail bin. When the life time of an item expires a new item (if any) is pulled from the bin of the respective position and the view is updated according.

The queuing algorithm can be changed at run-time to reflect different modes of priority - e.g. distribute top-priority items or show top-priority items as soon as possible. The standard algorithm is simple round robin (class: RoundRobinQueue).

[edit] Standalone Mode

The software application can me run from a URL and hosted on a web-server or run directly from disk. The standalone mode refers to the later and requires special considerations when setting up the system as defined in the following.

[edit] HTTP server

Flash can easily switch between reading from web services (via http) or directly from the file system because the UrlRequest object handles both URI and platform specific file paths. Though not recommended it is possible to serve media files from a local web server on Maemo. Here is how-to set it up to serve media and map tiles:

  • Setup light-weight web server, thttp, to serve media files
    • Add to "deb http://p.quinput.eu/debfarm chinook user" to /etc/apt/sources.list.d/hildon-application-manager.list
    • Install thttpd using apt-cache update and apt-get install thttpd
    • Change thttp configuration file in /etc/thttpd/thttpd.conf
      • Change chroot to nochroot
      • Outcomment nosymlinks
    • Create three symlinks in /var/www
      • 1) itinerary -> /usr/local/rlah/itinerary
      • 2) maps -> /usr/local/rlah/maps
      • 3) media -> /usr/local/rlah/media 
  Note: Best practice is to use the file system directly and avoiding the extra 
  overhead of the web server, http, and tcp/ip sockets.

[edit] Flash HTTP Policy Server

Note: This only applies to Flash applications served over HTTP.

To be able to access HTTP contents from another web-server a special configuration file has to be placed on the host server. The file is to be called "crossdomain.xml" and put in the root folder of the web-server. Flash will look up this file before fetching any other files from the server.

To allow all access from all places the contents must be: 

 <?xml version="1.0"?>
 <!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
 <cross-domain-policy>
   <allow-access-from domain="*" to-ports="*" />
 </cross-domain-policy>

[edit] Flash Socket Policy Server

Note: This only applies to Flash applications served over HTTP.

Similar to the HTTP server, socket connections has to be approved in a XML configuration file. In this case the approval file cannot be hosted on the HTTP server. Instead one either has to 1) setup a dedicated server on port 843 or 2) provide the file through the socket server that Flash will connect to. The later requires the socket server to be able to respond to a text-based protocol as specified in the Adobe Flash Security brochure. It is therefore much easier to simply run a dedicated light-weight service on port 843. 

 Important: All socket connections needs to be pre-approved using the crossdomain.xml schema. 
 Even sockets on the same server the Flash application is hosted from!

We provide a Python 3.0 script, which runs on both Windows, Mac, and Unix, to handle this task. The script is based on an official beta script from Adobe.

The script is started with this command where --file points to the policies in an XML file (e.g. crossdomain.xml): 

 python FlashPolicyServer.py --file crossdomain.xml

The contents of the crossdomain.xml needed to allow all connections is (notice the required to-ports attribute here allowing connections to all local ports): 

 <?xml version="1.0"?>
 <!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
 <cross-domain-policy>
   <allow-access-from domain="*" to-ports="*" />
 </cross-domain-policy>

[edit] Loading Map Tiles from file system

Modest Maps, the map engine of the Flash application, loads map tiles (images) from a HTTP server per default. A URLRequest object is created and run asynchronously. A Loader object handles the lifetime of URLRequest and dispatches events on completion. It is trivial to change the location - despite the fact the ActionScript documentation is unclear in this - to a local file system behavior. In our case simply change the location of the RLahMapProvider to a local path:

Unix version: 

 Providers[0] = new RlahMapProvider("Default", "/path/to/map/tiles/");

Windows version: 

 Providers[0] = new RlahMapProvider("Default", "C:\\path\\to\\map\\tiles\\");

If the Flash application is running via a URL, the Publish Settings must also be changed to "local access only". With local access only the socket based services will no longer work. Hence, the GPS and SQLite service will stop working.

If the Flash application is running from disk in "local access only" mode, network sockets will not work. If set to "network access only" sockets will work but only if the user has granted the file location, from which the Flash application is running, special privilege (see next paragraph).

To enable simultaneous file and network access without enabling development mode (which has other side effects) the user must go to Flash's Global Security Settings panel and add the file or folder to its trusted locations. The panel runs in the browser from Adobe's website.[3]

The alternative is to create a configuration file (*.cfg) file in the users ../#Security/FlashPlayerPolicy directory and specify a list of trusted locations.[4]

[edit] Tentative

  • Move from SQLite to SpatiaLite
  • Store usage statistics (has been seen, for how long, etc)
  • Advanced view of media files (concurrent video/image, fisheye, carrousel, grid-view etc)

[edit] User Documentation

[edit] MediaViewer Features

  1. Displays ranked list of media files (text, image, sound, and video)
  2. Can trigger on location events from GPS
  3. User can specify duration for each media file
  4. Can show images and sound concurrently
  5. Continuous nearest neighbor triggers slideshow events
  6. Pulls data from local database
  7. Sound and text can run a a separate background/overlay track

[edit] How-to Configure Tablet (Manual)

This tutorial is written and tested with N810 OS2008 (DIABLO_5.2008.43-7). The WIFI connection was used to fetch the data. 

  It looks complicated but it really isn't just do it carefully, 
  step-by-step, and drink a lot of caffeine.

[edit] Maemo setup

  • Update N810 to the latest firmware. Use the Nokia Update Wizard. Note: Runs only on Windows.
  • Setup WIFI connections.
  • Enable "maemo Extras" in Application Manager -> Catalogs

  • Install agps-ui (after install you must select your approximate location on a world map)
    • Run A-GPS Beta at least once.
  • Install openssh (may take a few seconds to finish as new PKI key pairs have to be created)
  • Install rootsh
  • Install subversion
  • Install unzip

  • Install python2.5 (use apt-get)
  • Install python-sqlite (use apt-get)
  • Install python2.5-gpsbt (use apt-get)
  • Install wget (use apt-get)

[edit] One-time software configuration

  • OS2008 no longer uses /etc/fstab but memory cards are still mounted without execute permission.
    • Open /usr/sbin/mmc-mount
    • Remove dmask and fmask. Append umask=000. This will give full permissions to the memory card.
    • There should now be one line in the file looking like this: 
  #!/bin/sh
  mount -t vfat -o rw,noauto,nodev,noexec,nosuid,noatime,nodiratime, \
  utf8,uid=29999,shortname=mixed,umask=000 "$1" "$2" > /dev/null
  • Update the boot-time graphics in /usr/share/icons/hicolor/scalable/hildon
    • qgn_indi_startup_nokia_logo.png is a small logo shown first at boot
    • qgn_indi_progressbar.png is the color and style of the progress bar
    • qgn_indi_nokia_hands.jpg is the full-screen images shown second and boot
  • Download code and data update shell script (rlah-810-update.sh)
    • Run the shell script as root (it may take a while to download and decompress media and map data)
    • Accept SSH certificate from la.remap.ucla.edu
  • Use the Flash Settings Manager's Global Security Settings panel to establish trust with the location of the RLAH directory. This can be either a file or network location (e.g. /usr/local/rlah). 
  If the Flash SWF file is compiled with network support only a Flash Policy server 
  can be used to grant access to external services. See the FlashPolicyServer folder.

[edit] Every-time software configuration

  • Run the rlah-810-start.sh script (this starts the GPS and SQLite socket servers)
    • Create S99rlah in /etc/rc2.d/ (see below for contents of S99rlah).
    • Remember to set file permissions on S99rlah to 777.
  • Run the Flash SWF file from it location in the file system (e.g. file:///usr/local/rlah/Application/rlahTablet.swf)

[edit] Performance optimization

The Maemo ships with an extensive UI framework. Getting rid of as much as possible is key to increase the available memory and lower the CPU load. A easy fix is to eliminate certain script bootstrapped at startup by Linux RC. 

  Important: After changing the bootstap list in rc2.d you 
  must power-off the device completely or the device will go into a reboot cycle.

This is the list of services needed to start the tablet (it will otherwise reboot infi). The listings were generated with: 

  ls -l | tr -s ' ' ' ' | cut -d' ' -f9-12 | sed 's/^/\* /'
  • K00zzinitdone -> ../init.d/zzinitdone
  • S12fb-progress.sh -> ../init.d/fb-progress.sh
  • S20dbus -> ../init.d/dbus
  • S20hal -> ../init.d/hal
  • S20osso-applet-display -> ../init.d/osso-applet-display
  • S21mce -> ../init.d/mce
  • S21x-server -> ../init.d/x-server
  • S22af-services -> ../init.d/af-services
  • S24dsp-init -> ../init.d/dsp-init
  • S25multimediad -> ../init.d/multimediad
  • S26esd -> ../init.d/esd
  • S26osso-systemui-early -> ../init.d/osso-systemui-early
  • S30ke-recv -> ../init.d/ke-recv
  • S45maemo-launcher -> ../init.d/maemo-launcher
  • S50af-startup -> ../init.d/af-startup
  • S52hildon-application-manager -> ../init.d/hildon-application-manager
  • S53dnsmasq -> ../init.d/dnsmasq
  • S54gpsdriver -> ../init.d/gpsdriver
  • S54libgpsbt -> ../init.d/libgpsbt
  • S54osso-ipv6 -> ../init.d/osso-ipv6
  • S54supllistenerd -> ../init.d/supllistenerd
  • S55ssh -> ../init.d/ssh
  • S56bluez-utils -> ../init.d/bluez-utils
  • S57btcond -> ../init.d/btcond
  • S58wide-dhcpv6-client -> ../init.d/wide-dhcpv6-client
  • S58wlancond -> ../init.d/wlancond
  • S59icd2 -> ../init.d/icd2
  • S60af-base-apps -> ../init.d/af-base-apps
  • S61osso-systemui -> ../init.d/osso-systemui
  • S99hulda -> ../init.d/hulda
  • S99mediaplayer-daemon -> ../init.d/mediaplayer-daemon
  • S99metalayer-crawler0 -> ../init.d/metalayer-crawler0
  • S99rlah
  • S99tablet-browser-daemon -> ../init.d/tablet-browser-daemon
  • S99ttyusb0 -> ../init.d/ttyusb0
  • S99zzinitdone -> ../init.d/zzinitdone

The following can safely be removed (e.g. to /etc/rc2.d/backup):

  • S41product-code -> ../init.d/product-code
  • S51hildon-desktop -> ../init.d/hildon-desktop
  • S70obexsrv -> ../init.d/obexsrv
  • S99alarmd -> ../init.d/alarmd


You are done!

[edit] Connect using SSH post-setup

First minimize the browser window. Do not exit it as it will cause it to reboot. Now open the terminal and type "root" and then "ifconfig" to get the ip address. This, of course, assumes that it is not in kiosk mode.

If the tablet is not in kiosk mode then either 1) lookup the tablet's IP on the DHCP server OR 2) use the the USB cable and telnet into the table on boot. The later requires however that a boot-time tool has been installed (e.g. initfs).

[edit] Getting the Client Running on a Workstation

[edit] Generate a Custom Map

To use a custom map with the RLAH software you must follow these instructions to create a new zoomable and dragable map for Modest Maps (our maps engine): [5]

After creating the map files create a ZIP file with the contents. The folder structure inside the ZIP file should be as follows: 

  maps/[name of tour]/

Inside this folder the ImageProperties.xml file should be and the folder with the map tiles generated my Zoomify (see tutorial link above). Name the ZIP file "maps_[name of tour].zip". Place this file on a public web server and point to it from the rlah-810-update.sh script.

[edit] Generate a Compressed Media Bundle

Use the Synchronize.py script to fetch the data from Gallery2 and download the images. Use the configuration file to specify the Gallery2 interface and the local folder to store the SQLite database and media files.

Next, create a ZIP file with the data. The folder structure inside the ZIP file should be as follows: 

  media/[name of tour]/
  media/[name of tour]/database

Inside the root folder place the contents in the media folder generated by the Synchronize.py script. Place the SQLite database (called rlah.db) inside the database sub-folder. Name the ZIP file "media_[name of tour].zip". Place this file on a public web server and point to it from the rlah-810-update.sh script.

[edit] Cloning and Running the Setup from a MMC

[edit] Pre-configure prototype device

  • Setup a clean tablet with all software and media.
  • Install dependencies 
 apt-get install wget
 mkdir -p /home/user/bin
 cd /home/user/bin
 wget http://www.nmacleod.com/nokia/bin/nupgrade.sh
 wget http://www.nmacleod.com/nokia/bin/tar
 chmod +x nupgrade.sh tar

 apt-get install e2fsprogs
  • Partition the external memory card 
 apt-get install e2fsprogs
 umount /media/mmc1
 umount /media/mmc2
 sfdisk /dev/mmcblk1 << EOF
 1,15000,6
 15001,,
 
 
 EOF

 cd /home/user/bin
 wget http://fanoush.wz.cz/maemo/initfs_flasher.tgz
 tar zxvf initfs_flasher.tgz
 cd /home/user/bin/initfs_flasher/
  • Flash initfs 
 cd /home/user/bin/initfs_flasher/
 ./initfs_flash
  • Configure initfs to show boot menu on start 
 chroot /mnt/initfs cal-tool --set-root-device ask:mmc2
  • Run the custom clone script 
 cd /home/user/bin
 ./nupgrade.sh 0 exernal
 ./nupgrade.sh 1 exernal
 ./nupgrade.sh 2 exernal
 ./nupgrade.sh 3 exernal (NOTE: Skip this step if you are installing a custom disk image)
 ./nupgrade.sh 4 exernal
  • Lastly set initfs to boot from the ext mem card by default 
 chroot /mnt/initfs cal-tool --set-root-device mmc2

[edit] Copying the MMC 1:1 using Unix

  • Use dd to make an exact copy of the source MMC (e.g. dd if=/dev/mmc1 of=/dev/mmc2)

[edit] Setting up cloned devices (option A)

  • Insert cloned MMC.
  • Install initfs boot manager 
 cd /home/user/bin
 wget http://fanoush.wz.cz/maemo/initfs_flasher.tgz
 tar zxvf initfs_flasher.tgz
 cd /home/user/bin/initfs_flasher/
  • Flash initfs 
 cd /home/user/bin/initfs_flasher/
 ./initfs_flash
  • Configure initfs to show boot menu on start 
 chroot /mnt/initfs cal-tool --set-root-device mmc2

[edit] Setting up cloned devices (option B)

  • Boot into the internal flash
  • Download image or file bundle (e.g. tar.gz file)

For file bundles:

  • Mount MMC 
 mount -t ext2 /dev/mmc.... /media/mmc1 (if mmc1 is already in use unmount it first)
  • Extract the bundle onto the mmc

For disk images:

  • Use dd to copy it straight to the mmc 
 dd if=image.bin of=/dev/mmc2
Retrieved from "http://wiki.urban.cens.ucla.edu/index.php/Remapping_LA"