Geotwin: Test Demo

1. Geo Twin 1

   • CDN load of test code, some reading up.
   • 10,000 points of lng/lat data from modified csv.
   • Loaded carto basemap and hexbin data (d3 implementation)
   • TO DO: Clean-up of data structures for presentation.
   • TO DO: Add timer to play through data. Think about UI and playback functions.
   • TO DO: Add secondary line layer. Think about animation
   • TO DO: think about lazy load screen for data, as the files are huge.

2. Geo Twin 2

   • Simple animated play timer (on load) of 120 15-minute chunks, updating data every 200ms. No UI included yet!
   • Plotting of data points based on activies.csv file. Each 200ms snapshot is total number of lat/lon points that fall within the 15min window.
   • To see whether the event falls within the 15 min chunk, made some assumptions to come up with the logic:
     • event start time <= current time + 15 mins AND
     • event end time >= current time
     • Assumption: Event end time must be more than event start time. Both fields are required!
     
   • This dataset seems to be a bit more than the one in the video.
   • As a comparison, if I just use event start time >= currentTime AND event start time <= currentTime + 15min, (i.e. between the window) the resulting dataset is pretty different and seems more similar to the video. Left this in as code, just uncomment if want to run.
   • Research: Spent time doing some reading. Deck.gl uses d3 hexbin aggregator, which just passes in a primitive array of lat/lng points. To show the result in the video (only aggregate and show the highest type), I suspect need to extend the d3 hexbin aggregator (i.e. look at and modify / extend the d3 hexbin code and not just use the library). Argh.
   • Research: Not that many examples for deck.gl, especially on transitions. Most of the transitions seem to be fly to a geospatial location. Few examples on linear interpolation when dataset is called. More research / reading to find out if intepolation is easy or tedious. Suspect the latter.
   • Considered easier methods like only passing in an event row if it has the highest number of events in that class per lat/lng, but likely logic wise does not seem correct. Need to aggregate first than rank.
   • Anyway by showing only the max type of a class, the visualization may be inconsistent. What happens if several types have the same equal value? Which one to show?
   • As the dashboard is for exploratory analysis, I think having some UI to split and pick different types might make more sense. More experimentation needed.

3. Geo Twin 3

   • Base UI added.
   • Picker to choose different event types. Event types dynamically generated from the dataset (unique type entries). Lodash library added for some data manipulation.
   • Rewrote timer using base JS setInterval rather than d3.timer. Simple play button (restart from earliest time) and stop button.
   • Information bar - print out current time window. Also total number of data points plotted on screen.
   • Legend added. SVG, done using D3.
   • UI gives the ability to pause, unwind, and step through the dataset. The dataset, as long as you load any new data in the activities.csv format, it should work. Not using the geometry field.
   • Comments: Most of the data is of the type "home" - which means the person is at home?
   • Comments: The most interesting thing that can be seen from the dataset is the aggregation of data points at tertiary institutions (on a Sunday!)
   • Comments: Reducing / aggregating the number of types in the data might make for a more meaningful visualization.
   • E.g. all the school types aggregate into one tag, etc. Some tags e.g. "w_1030_0630" may not be needed if the tags refer to time bands, given that we're already plotting by start and end times.
   • Consideration: For person, activity and facility ids there are ways to aggregate the data, interesting exploratory analysis can result. Need schema to see what the fields are.
   • Consideration: For niceness, create a nice timeline slider for UI interface. Will only polish if have time.

4. Polished: Geo Twin 4 (3hrs)

   • Polishing: Quality of life interface options added.
   • Pickable tooltip from the deck.gl library added. Prints out data point values of each hex cylinder.
   • Wrote in own slider timeline, and hooked everything properly with a play / pause button and a scrubabble timeline.

5. Geo Twin 5

   • Added triplayer.json, and did a major data wrangling of the data.
   • Every line is a series of 3 4-variable arrays (labelled as coordinates), and one of the sets of the data is exactly the same as the other, so not necessary.
   • The structure does not follow the geojson schema, so I cleaned it up a bit via code into the structure I wanted.
   • Assumption: For the 4 variable array I don't have the published schema, but here are the imputed fields:
     • 1st var: longitude
     • 2nd var: latitude
     • 3rd var: elevation(?) All values are 0
     • 4th var: seems like UNIX timestamp
   • 2nd point timestamp is always less than the 1st point, so I have assumed that this is the origin, and the 1st point is the destination.
   • For the lines I have plotted only when the event is fired, i.e. only when start time > currentTime and starttime <= current time + 15 mins.
   • Otherwise some of these lines and the events they represent will clutter the screen as some take a pretty long time (over hours).
   • Consideration: Rather than 2 point lines, if we have more granular data I suspect we'll get a more interesting and useful visualization. Constantly sample where the agent is at over a time period and plot their position on the map.
   • This might give us more insights into the data and pattern of travel.
   • Because currently by plotting all the lines in the above way it it may not be useful in giving a higher-level holistic insight. Or with this specific dataset it is not obvious.
   • Consideration: We should consider having interface options for different visible layers (a form of faceting) so that we don't clutter the viewport at any one time and it is easier for the user to get a sense of the data.
   • TO DO: think about using tripLayer for animation of line paths.

6. Geo Twin 6

   • Inidivdual agent file is huge 20+ MB, so just linking to a video (2MB+)
   • Tried tripslayers for animation of the line paths. But because of the way I am redrawing the layers every 200ms (representative of a 15min block) using my own js interval function, I think this clashed with the way deck.gl tripsLayer does the animation.
   • To check, gotta really go back and look at the base code for how this layer is implemented in deck.gl.
   • So I tried another experiment, which I think is more accurate anyway. The tripslayer path lines (representing an event) has a start and end timestamp, and this often is measured in hours rather than minutes.
   • If you plot lines throughout the whole duration, it creates a ton of visual clutter, which I mentioned earlier as a problem.
   • So instead of drawing lines, draw points representing a person. If you interpolate between the two points of the line data given, you can simulate a person's position given you have the start / end timestamp of the event.
   • I wrote in my own linear interpolation function to animate the event between all the 15min steps for this version.
   • This I think is a far better solution than v5, which draws the line when the event started to fire (and does not take into account fo the end time) because it cluttered up the whole screen.