Enipedia Maps

Feel free to add any feature requests, bugs, etc to either this page or the discussion page.

Background

While we've created a Google Earth Visualization for the data on Enipedia to allow people to navigate it more easily, this has limitations in terms of interactivity, such as being able to select different subsets of the data. The interface below shows some of the work in progress that we're doing on addressing this and create an interface that users can tune to there needs. The code heavily borrows from what Nono has done to visualize data from Enipedia and other sources.

Functionality

Currently, the following functionality is implemented:

• Upon clicking the url, Google Maps comes up with a menu displayed on top of the page
• The idea is that you first select a country
• Subsequently you are able to filter various combinations of power plants by their fuel types.
• We allow you to turn on or off specific fuel types, or complete categories of fuel types:
  • fossil fuel (thermal power plants)
  • nuclear (thermal power plants)
  • renewables (wind parks, solar, wave, biomass thermal, geothermal etc.)
• In the menu, the numbers under the icons show the number of power plants found for each fuel type for the particular country selected.
  • Please note some double counting may occur (across different fuel types) if a plant uses several fuel types.
• As you start typing in the owner, it will try to autocomplete the name, based on entries contained in Enipedia.

(See map as whole webpage)

• The data used for the map is based on the live data on the wiki. Any changes to the wiki data will show up on the map once you refresh the page.
• You can click on the Advanced Menu to load all of the power plants in the current viewing window.
• Whenever markers are shown on the map, the Advanced Menu also contains a link to download the data in an Excel format.

Source Code

The source code is now up on github at https://github.com/cbdavis/enipedia-maps. This will likely evolve and branch as we figure out what's feasible and experiment with different options. Feel free to experiment with it and list bugs or suggestions for improvements or new features.

The code runs as an HTML page, so people can run it on their own server, or locally through a browser. The main dependency is that the code talks to a SPARQL endpoint on our server, so you need to have an internet connection to get it to work. The URL for the endpoint will likely change as this is part of an effort on Upgrading Enipedia.

The code is interesting for open data projects as it is able to query a SPARQL endpoint and then display and filter the data. Further functionality to be integrated can be found here.

Sandbox

A sandbox for experimenting with the code is here since the map is not wiki-enabled. This may contain some glitches, especially when combining country and owner. New functionality tested in the sandbox usually makes it into the github repository and onto the site here.

Issues & areas for improvement

Data

• View/edit links do not work for powerplants whose name contains & or ' characters (no page for the former, bogus page for the latter).
  • This is one of the issues to check after we upgrade the site. One of the modifications made is to change how SMW encodes URLs in exported RDF. These modifications can be currently checked via the RDF export on the test site: http://enipedia.tudelft.nl/enipediaTesting/index.php/Special:ExportRDF/Vitry_Powerplant

Code

• The download link shows up even if there's no data found for the query.
• References to external javascript libraries cause security warnings. On one version of Chrome, it was observed that nothing loaded, and it wasn't obvious that content was being blocked. For the version of the maps on Enipedia, we should include the libraries locally. I'm not sure if there would be a solution for the scraperwiki version.
• The code for country selection doesn't load all the time. Sometimes the page needs to be refreshed.
• Need to upgrade SMW so we can use the sfautoedit API call to update wiki data directly from the map. Easy first step is by setting markers to be draggable so that people can update the coordinates. But this can also be dangerous, especially with many markers cluttering the map space, and may at least require a confirmation. It would also be unwise to use it until we can achieve synchronisation with the wiki, since it would imply editing data you don't know the current state of.
  • There should be some sort of explicit editing mode/sequence that helps to prevent confusion. Need to think about how to make people remember to save changes, how can they undo, etc.
  • sfautoedit is working with the current software but it seems to be tied to forms which implies that you cannot automatically edit fields such as primary_fuel_type or latitude/longitude that are in the template but not in the form. Another downside is that if you don't specify the point property during the edit it gets lost during the edit, just as when the map fails to load.
    • latitude/longitude will probably still be derivative properties of point since (it seems) that the maps interface in Semantic Forms can only write to a single property. Conversion of point values to lat & lon is done automatically when the page is saved. The fuel_type and primary_fuel_type do need to be refactored to something more sane, perhaps like primary_fuel_type (1 value), secondary_fuel_type (1 value), other_fuel_types (multiple values). This would have the added benefit of getting rid of the permutations of results that sparql queries generate when one plant has multiple values for fuel_type specified.
      • (fixed, also slight rounding/truncation occurs, but this has not been seen to introduce noticeable differences) latitude/longitude do not actually get updated on save when point changes (see Malassis_Powerplant or Motz_Powerplant)
        • It's not clear yet what's causing this. Examining Malassis_Powerplant and Motz_Powerplant shows that the lat/lon seem to be truncated versions of the value set for the point. It's strange the point would be set, and not the lat/lon since these are all set from the point field in the template. Also some tweaks have been made to the cache settings to (hopefully) fix an issue where the point on the map was not updated when the coordinates were changed. The combination of these may be causing the problem, or there may be a deeper issue.
        • Special:ExportRDF/Malassis_Powerplant shows that the values for lat/lon are directly based on the value for the point. However, in virtuoso, these get truncated to what is shown in the factbox at the bottom of the page. This seems to indicate that Special:ExportRDF may not use exactly the same code as that which synchronizes the RDF on the wiki with the triplestore.
        • Is it possible to remove the latitude/longitude fields from the template if the RDF properties are definitively set from the Point field? Having them remain with their outdated values can be confusing.
          • The majority of pages have been cleaned up via a bot script. One more pass is needed to make sure all pages have been fixed (seems to be memory leak on RSemanticMediaWikiBot code).
      • (fixed) There are also a number of power plants that still do not have latitude/longitude at all, though they should have been updated during upgrade. Those just do not show in the latest sandbox version. For example, looking at Guadeloupe island in the French West Indies displays only 3 plants while there should be at least 17 if latitude/longitude were not mandatory.
    • We also need to see what happens when people edit a large number of points. If you edit 10 points, the code behind the scenes will save data to 10 pages. If this takes a long time to run, it would be possible to set up a queue system with a bot to process the edits.
• A somewhat related concern is that there are instances of power plants occurring at the same exact coordinates due our use of geocoding for power plants that did not have any coordinates specified in the original data. We may want to consider using a bot to offset the points so that people can differentiate between them when they are all the way zoomed in. In these cases, we know they're incorrect anyway, but (hopefully) in the correct general region, so this shouldn't mess up things. Modifying data because of a display issue is however not the best practice. Using an advanced marker as in the sandbox may be a better solution, though it could be a bit smoother.
• Values for data derived from the carma.org data set should be optional in the sparql queries that get data for the map. This way if we only know the name and location of a power plant it will show up on the map. Since the icons are scaled due to the output of the power plant, we need to think of some default value to use if nothing is known. Eventually the maps interface should evolve (split off?) into a tool that allows for highlighting data with missing property values.

Appearance & Enipedia

• We need an extra icon to distinguish Fuel Oil from Diesel Oil (Gas Oil in English mostly...)
• Width of control panel is too wide for embedding. Should look into moving the google maps controls out of the way.
• Control panel gets in the way of some of the icons. Fix by setting slightly bigger bounds?

Functionality

• The SlickGrid javascript library could be useful for integrating an interactive tabular view that could allow for another way of filtering the data. A very cool example is here. This could be adapted to allow people to explore other data sets, such as the EU-ETS. Functionality such as grouping would allow selection of plants owned by a set of companies.
  • We've got a proof-of-concept running - this needs to be placed on scraperwiki. With a test on EU-ETS data, it's quick for navigating data (once it's loaded), but grouping can be slow. For NL data, grouping by account holder takes a few seconds. Doing the same for the whole dataset takes considerably longer. See here for development notes.
• For some of the ideas below, may want to split the functionality into two interfaces - one simple default (similar to what we have now), and another for more advanced types of filtering. See the code on scraperwiki for initial work on being able to switch between a single and advanced menu.
• Add selection on capacity installed (for example > 1 MW, > 10 MW, > 100 MW > 1000 MW
• Add selection for "status"
  • I don't think that we have that much data on power plant status currently.
• Add a link to this view on country pages, of course viewing only that country data
• Add intuitive clicks -- navigate the map, click on a country and get the power plants
  • An easier option is to only select by bounding box (requires prop:Point to be split into lat and long properties). Being able to click on a country requires that we have polygons defined for country borders and then have some way to calculate which polygon the mouse is in. I'm not sure if there's an easy way to do this within Google maps. The bounding box approach would be more robust as it would allow people to get data from different zoom levels.
• Match Google's map functionality -- navigate to a place and show the power plants around it, depending on your own selection
  • or like Analytics functionality - clickable world map with statistics, with "power plant footprints" displayed, as shown on country pages
• Show selected statistics (again -- set by user preference?) on the view
• Add csv download - we have the SPARQL query for the data already, just change the output format. The data is also contained in a list in javascript as well.
  • See discussion here and here. It seems that the most robust solution is to just go through virtuoso which has an option to export to Excel. Trying to dump out data from javascript doesn't seem to be equally supported across all browsers.
• Click button to get top x powerplants within current viewing window.
  • Should limit this to a range of zoom levels so that people don't try to get everything at once and crash their browser (is it possible to load everything at once and still have the map function?).
  • The code on scraperwiki currently is set up to get all of the power plants within the current map bounding box.
• Add ItoWorld map tiles, based on code here. The version on scraperwiki allows for this.
  • This functionality is currently commented out in the scraperwiki sandbox. The main issue is that we should use a background (OSM Mapnik?) that allows us to very clearly see this layer. It's hard to see with the Google Maps satellite tiles. As an advanced option, we should allow people to show tiles from different mapping services.
• Something with D3.js and/or Crossfilter?
• Use circles to indicate emissions/output similar to what we've done with the Google Earth visualization - does this work for 1000's of points or should we use clustering instead? Could also just scale icons into different bins for a cleaner interface by using MarkerImage properties.
  • The current version scales the icons quite nicely. As an advanced option, it would be nice to choose which property the icons are scaled to.
• Allow for viewing external data sets separately from Enipedia data. Also allow for comparisons, such as shown here and here.
• Information display in popup windows can be improved - may want to try running DESCRIBE queries to get all the data related to the plant. This all depends of neede use: when comparing two datasets, having a small set of equivalent properties is more effective. On the other hand, if you use the map as an alternative interface to explore data, you may indeed want to see a more complete summary.
• Eventually the text field with autocompletion to query by owner company would be aware of which companies are owned by other companies, i.e. if you select Vattenfall, it would also show the power plants of Nuon. We don't have the data needed to do this yet, but if we did, a query using a property path would be able to traverse the ownership hierarchy.
  • The latest CARMA release (v3) seems to use top level company names (groups) instead of subsidiaries as previously. This could help in identifying companies from the same family. Perhaps set a parent company property?