CSC400-GeoTracking Spring 2011

From CSclasswiki
Jump to: navigation, search



Students

2011 IS LEA1.jpg

2011 IS LEA2.jpg



Prof. D. Thiebaut, Advisor

Meetings

1/24/11

  • Informational meeting
  • Purpose of the IS: continue the project from CSC220

To do for this week

  • Wait for individual schedules to settle
  • Read up on the concept of discrete event simulation
  • Please list the good references you find in the reference section below

2/3/11: Assignment

--Thiebaut 18:58, 3 February 2011 (UTC)

  • (No meeting this week... Snow days...)
  • Mini assignment for next week below

Aigerim and Lindsey

Imagine that I have a list of Longitude Latitudes and times in the following format:

<time>Start 7:50 AM</time>
<coordinates>
-72.69459750750001,42.34444296000000
-72.69435600000000,42.34400928333333
-72.69431068333333,42.34405935000000
-72.69446536666666,42.34410188333333
-72.69409073333333,42.34385796666667
-72.69392793333333,42.34376685000000
-72.69364926666667,42.34365636666666
-72.69293845000000,42.34334739999999
-72.69270220000000,42.34304800000000
-72.69264033333334,42.34287956666667
</coordinates>
<time>Start 8:10 AM</time>
<coordinates>
-72.69265161666668,42.34280600000000
-72.69287370000001,42.34265043333333
-72.69309221666667,42.34251263333334
-72.69328463333333,42.34241403333333
-72.69342835000000,42.34232905000000
-72.69359670000000,42.34220988333334
-72.69377464999999,42.34210050000000
-72.69396520000001,42.34200168333333
-72.69419445000000,42.34190578333333
-72.69439651666667,42.34189388333332
-72.69453363333334,42.34204025000000
-72.69470221666667,42.34222191666666
-72.69501901666666,42.34270666666666
-72.69512208333333,42.34282508333334
</coordinates>
<time>End 8:50 AM</time>
  • Write a program in Php that will output a list of triplets (latitude, longitude, time) where the time is linearly interpolated from the time information available in the
  • Start writing a discrete event simulator in Php that will take 2 list of (latitude, longitude, time) triplets and that will output them in the correct order of time.
  • Feel free to work on this in programming-pair mode, do both, or each one pick one problem and the other person the other. That's fine with me.

Betsy

  • You're going to start at the other end of this project.
  • Search the Web for Processing projects (or Java) that include source code and display either:
    • moving dots or circles or images that leave a trail
    • heat maps
    • graphics with a menu system allowing one to modify the behavior of the graphics.
  • At the same time, search for Web templates such as squarespace.com where we could find sophisticated sets of pages we can use for doing
    • student registration
    • selection of parameters
    • display of tracks.
(It's okay if it is a system that is not free. It's a good exercise for you three to create a set of Web pages for a project and learn how to do this for your own business-oriented service/project later.)

2/9/11: Assignment

--Thiebaut 14:42, 10 February 2011 (UTC)

  • Aigerim:
    • Look at ways to get your code to get data from the database. It will become a tunnel that Lindsey's code can use to get trace data. Ideally your program should go get a trace (or several) from the database, add time information to each point, and make it available to Lindsey's code. You and Lindsey have to decide on the format of the interface. Documentation is important!
  • Lindsey:
    • Implement a Processing sketch with the queue and a class defining a trace. Each trace obtained from Aigerim's code will be held in an object issued of this class.
    • Implement the queue that holds the tuples representing events in another class.
    • Demonstrate what you were able to do next week on the projector.
  • Betsy:
    • Keep looking for templates
    • Keep looking for hosting solutions as well
    • Start creating a new wiki page with a survey of what you find. You may want to look at my survey of data visualizations as an example of what you could do (http://cs.smith.edu/dftwiki/index.php/Visualizations:_Algorithms)
    • Investigate heat-maps, both as ways to model them mathematically and ways to model them in software.

2/16/11 Meeting and Assignments

Action Items

  • Betsy
    • Continue with research. Show us some interesting stuff next time so that Aigerim and Lindsey will be jumping up and down with desire to code it!
  • DT
    • Put the code from CSC220 on the class wiki page
  • Aigerim
    • Put a form together with some Php to put a trace in the database, such that the coordinates and interpolated time are also included on each line.
    • Use a format for the time that makes it easy to compare to other times, and easy to get information such as the day of the week, or the date. The number of seconds since 1970 might be a good way to go.
  • Lindsey
    • work in Processing (or Java)
    • Write the class for the trace data
    • Implement the event queue in a class
    • Run a simple test with 2 traces and have the event simulator output the (x, y, t) triplets of each trace in chronologic order.
  • LEA
    • Work on the server
    • Make it such that we can ssh to it
      • Need to do something with firewall, I couldn't get it to work -- Betsy 2/16
    • Use the Synaptic Package Manager and install mysql on the server
    • Install Php
    • Install emacs
    • Installed java default-jdk -- Betsy 2/16


2/25/11 Meeting and Assignments

OaklandCrimeSpotting.png


  • We are going to spend a week studying this and figuring out the answers to some of the following questions:

Action Items for LEA+DT

    • What language is it programmed in? Javascript?
      • Javascript
    • How open is it? Is it open-source?
      • Yes
    • Can we upload our data easily?
      • Yes. The data is uploaded manually by users to a database through a form on CrimeWatch. The application processes that information and creates a visualization.
    • Can we display tracks instead of points?
    • Can we do animation?
      • No. Static representation only.
    • Are the widgets programmable?
      • Yes.
    • Can we easily remove/add widgets?
      • Yes.
    • Can we host this on our own server?
      • Yes.
    • What data format is accepted (KML)?
      • The data is uploaded via a PHP form in plain text. No files are used. Webpages displaying KML and GeoRSS feeds can be mapped.
    • How do we do [KML Data] ====> [Map] with this system?
More Information on CloudMade and Other Mapping Programs
  • Modest Maps displays tile-based maps in Flash and Python. Provides links to JavaScript based mapping projects that could provide ground work for mapping our tracking project.
    • OpenLayers A open-source JavaScript mapping framework.
    • Mapstraction A JavaScript mapping API wrappers. Works with GeoRSS and KML. Open-source.
    • OpenZoom A flex library for tile-based image viewers. A toolkit for the Adobe Flash Platform.
    • UMapper A Universal ActionScript 3.0 Mapping API. Has integrated data from OpenStreetMap, CloudMade, and others. Expensive... $2500 per domain per year.
  • CloudMade A open source JavaScript mapping framework.

CloudMade Example.pngCloudMade Example With Track.png

3/4/11 Meeting and Assignments

Action Items

  • LEA: write down answers for all the questions above, please!
  • DT: Create a lea user with access to all 220 databases on the MySQL server
  • Aigerim:
    • Set the day of the week in the database to be that of when the trace was captured
    • Figure out what language is best to have a CRON job read mail messages from a mail account. It might be that people have solved this for gmail. Python, Php, or even Perl solutions might already exists. Figure out what seems easiest
    • Otherwise we'll have to install a mail server on our Ubuntu box
  • Lindsey:
    • Continue work on the Discrete Event Queue
    • Flesh out the URL listed above so that we have a better idea what they show, what they do, how they connect to what we are after.
    • Make the program that uses the queue issue dequeue requests in a loop, and every time a request is made, a time stamp is given to the Discrete Event Queue. The queue will use the time stamp as a marker and will output all the events from the different traces it keeps that are older than the marker
    • For example, assume we have only one trace with event t100, t102, t103, t105, t110, t112, where the index is actually a number of seconds. If a request to dequeue is sent with Time Marker 99, then the queue does not output anything. If the next time a request to dequeue is made but with a new marker equal to 106, then the Discrete Event Queue will output t100, t102, t103, and t105.
    • See if you can connect this queue to a processing applet showing the actual trace
  • Betsy
    • Figure out how we can use R along side Processing.
    • See if you can show an example of a Web page with both side by side
    • See if you can answser the following questions:
      • Can R read data from a URL?
      • Can we put a Php tunnel as the URL, allowing us to pass parameters?
      • How can we pass the parameters to R from the Web page?

4/1/11 Meeting and Assignments

  • Short term:
    • Next meeting is Monday, at 7:00 p.m.. Just a 10-minute meeting to discuss the overall organization of the project as you all three see it. We'll use the board and draw a diagram of the process and the different elements.
    • I will try to print the poster on Tuesday. Let's make sure it's done by Monday night.
  • Longer term:
    • Betsy will work on pure R stuff, defining exactly what type of input she needs to generate heat-maps of the campus. The trick will be to generate a map that is to scale and can overlap exactly with an image of the campus.
    • Lindsey will work on getting a smooth demo that runs in a loop on the laptop.
    • Aigerim will work on interfacing Php and Processing to Betty's R program.

4/4/11 Meeting



LEA Block diagram javascript.jpg


4/8/11 Meeting

Action Items:

  • Aigerim: Working on her application for Google Summer of Code
  • Betsy
    • Will take Stanford example
      1. recreate and make the example work as it is presented
      2. feed KML data in text area ==> generate heat map
      3. will interface with LA for data format of input (already in progress)
  • Lindsey
    • Add a title page to demo
    • Eventually, the digital clock should include day of the week, plus date in full (and time)
    • Define what the dimensions of the Heat Map should be (was done by the time the meeting ended):
      • dimension of campus map = 715 x 695
    • Find Longitude/Latitude info for top-left point of Smith map, and for bottom-right point of same map, and pass it on to Betsy (was done before meeting ended).
    • (Note to Lindsey: why don't you add this info here so that we can find it easily if needed? Thanks!)

4/14/11 Meeting

  • We set the laptop up with Lindsey's demo for the poster session.
  • Lindsey will pick up the laptop and bring it to the campus center for Saturday morning
  • Betsy will be in charge of the posters (both the "Smith Walk" poster and Elizabeth Do's poster)
  • Betsy showed her current research on David Kahle's research posted on Github.

Action Items for Next Week

  • Meeting will be Friday 4/22/11 (4 = 2 + 2, 2 = 1 + 1)
  • Betsy will try to make the code on Github work with the map of Smith...
  • Lindsey will incorporate the Applet in a Web page and create a simple selection scheme...


4/22/11 Meeting

  • Good overview of where everybody is.
  • Aigerim presented Javascript+JQuery interacting with Processing Applet
  • Lindsey found a missing coma in form requesting traces belonging to a time-frame defined by two times entered in seconds since Jan 1, 1970.
  • Betsy showed a Heat Map of Smith done in R

Action Items for Next Week

  • Meeting will be at 2:30 (possibly earlier)
  • Betsy will package her R script and give it to Aigerim to install on server. Betsy will also define the data format required by the R script to work. Aigerim will make the Php tunnel
  • generate a data file in the format required by the R script
  • call the R script so that it can generate a new image of the heat map.
  • Lindsey and Aigerim will work on integrating the Processing applet with the discrete event queue with the javascript menu.
  • Lindsey integrates Betsy's last heatmap of Smith in the presentation loop.

Goal for last meeting of semester

  • A form allowing user to pick different options and display the dynamic evolution of the traces
  • The same form will display an applet and an image of the heat map corresponding to the applet.



References/Resources

Poster Printing

Discrete Event Simulation

  • Provides a way for us to sequence coordinates in chronological order, regardless of which track they correspond to. Discrete event simulation normally implements a queue that is sorted in chronological order. In our implementation, the queue will be resorted each time a new coordinate point is added to it. This will ensure the queue will remain sorted chronologically.

Team Rain's Project Files














Poster Page

Preliminary version: File:CSC400Poster.pdf


Color schemes: Collaborations.png

Results

Code

All code is given in File:Lea.zip

Project structure

CSC400.jpg

  • uploadForm.php - a form that takes user's email and a KML file as input and passes them to uploadToDatabase.php via JavaScript
  • uploadToDatabase.php - checks whether the user email is in the database, and if it is, inserts the data from the KML file into database 220a_project.traces in the necessary format, calculating time for each point in the trace.
  • index.php - The main page that displays the map applet and the menu with categories. The applet is loaded using data in file uploads/kmldata.csv. The form values are passed to getKmlData.php. If it is successful, the applet is reloaded with the new traces using JavaScript by erasing the applet history.
    • Current issues - The applet does not display the data in kmldata.csv. It works with one file; but it does not work with files that follow the exact same format. Sometimes when the applet tries to load the file it freezes, crashing the browser. We couldn't find the reasons of this behavior.
  • basic.css - stylesheet for project pages.
  • getKmlData.php - gets form input from index.php and queries the database 220a_project to retrieve traces that match given categories. For each trace, it gets the "coordinates" field, and writes it into the file /uploads/kmldata.csv, appending trace ID at the end of each line in order to differentiate them.
  • Processing files - take the file uploads/kmldata.csv and display them using a Discrete Event Queue.