Administrators

A guide for the administrators of eMonocot portal

Identification key

Q. How can I import a new key?

A. In the portal a key is a type of resource. Therefore to import a new key, you have to follow the instruction for creating a new resource. As for any other resource you need to be an administrator in order to perform this task. If you are not an administrator, look at the section "Identification key" in the "Contributors" portal guide.

Q. What is the difference between 'Harvest' and 'Harvest changes'?

A. Both jobs harvest a resource (file) from somewhere on the internet. "Harvest Changes" issues a Conditional GET with an "If-Modified-Since" header set to the date that the resource was last harvested. If the resource has not changed at all, the server should respond with the code 304 (meaning "Not Modified"), and the job will terminate. Otherwise the job will continue and the resource will be harvested. "Harvest" issues a GET, and will harvest a resource whether it has been modified or not.

Q. How can I update the content of an existing key?

A. If you are an administrator, once you are logged in, go to the resource page of the key you are interested in and click on 'Harvest' or 'Harvest changes', depending on the type of update you are interested in.

Q. How can I edit the information about an existing key resource?

A. If you are an administrator, once you are logged in, go to the resource page of the key you are interested in and click on 'Edit this resource'.

Organisations

Q. How can I create a new organisation?

A. Login, go to the general organisations page (www.e-monocot.org/organisation) and click on ‘Create new organisation’ button.

Q. How can I edit an organisation?

A. Login. Go to the organisations page (www.e-monocot.org/organisation). Search for the organisation which you want to edit (or page through the list). Select the organisation, then click "Edit this organisation"

Resources

Q. How do I create a new resource?

A. Every resource is associated to an organisation. Once logged, if you are an administrator of the portal, go to the organisation page related to the resource you want to create and click on 'Create a new resource' (in the section "Resources" at the bottom of the page).

Q. How can I edit information about a resource?

A. Login. Go to the resources page (www.e-monocot.org/resource). Search for the resource which you want to edit (or page through the list). Select the resource, then click "Edit this resource"

Q. How is the date formatted?

A. The date is an ISO 8601 format Date-Time without timezone information. It is YYYY-MM-DDThh:mm:ss . All datetimes are in the timezone of the server hosting eMonocot, not the local time of the user.

Q. How can I harvest the resource periodically?

A. On a resource page: select the desired Harvesting interval (daily, weekly, monthly or yearly); check "Harvest periodically"; submit the page.

Q. What is a ‘Job Id’?

A. Resources are processed using a batch processing framework called "Spring Batch". The Job Id refers to the Spring Batch Job Execution Id. Administrators should not need to change this as it is updated automatically when a job starts.

Q. Which parameters can I add?

Darwin Core Archives

Setting up a new scratchpad resource

1. Once you have created a new organistion to represent the scratchpad

2. Scroll to the bottom of the page where it says "Resources" and click on "Create a new resource"

3. Give the resource a name which makes sense such as "{Family} Darwin Core". The URL should be http://{family}.e-monocot.org/dwca.zip and the type is DwC_Archive.

4. Click "Submit"

5. Now search for the resource you just created - type "{Family} Darwin Core" into the search box and click the blue button with the white magnifying glass icon

6. Click on the name of the resource to navigate to the resource page

7. Now click "Edit this resource"

8. Use the bit at the bottom of the page to add the following parameters - for each parameter, add the parameter to the box, then click "Add Parameter". Don't fill in the value for a parameter until you have finished adding the parameters to the job as clicking "Add Parameter" submits the page but does not save any changes you have added to the job

  • taxon.processing.mode CHECK_TAXA
  • image.processing.mode IMPORT_IMAGES
  • reference.processing.mode IMPORT_REFERENCES
  • typeAndSpecimen.processing.mode IMPORT_TYPE_AND_SPECIMEN
  • distribution.processing.mode IMPORT_DISTRIBUTION
  • description.processing.mode IMPORT_DESCRIPTIONS
  • vernacularName.processing.mode IMPORT_VERNACULAR_NAME

9. If the resources covers a taxonomic family, subfamily, tribe or subtribe, add a parameter with the name of the taxonomic rank, and the value of the name of the group e.g. if the resource covers the family Acoraceae, add the parameter

  • family Acoraceae

Then click "Submit"

10. OPTIONALLY: To schedule this resource to be harvested e.g. weekly: Click "Edit this resource" and then check "Harvest periodically" and set "Harvesting interval" to WEEKLY, then click "Submit"

Allowed Parameters for the DwC Job

Generally, sources that produce all of their data in a single archive and are taxonomically scoped should have description / distribution / vernacularName / measurementOrFact / identifier set to IMPORT_* as the default is IMPORT_DONT_DELETE_GLOBAL and this will not delete records outside of the scope of the group. For non-taxonomically scoped resources such as floras, or resources which encompass all monocots, but which are sparse such as the IUCN import or the SRLI monocots it is important to set the taxon.processing.mode to SKIP_TAXA which will not check the taxa but will mark the taxa in the archive so that they are indexed properly afterwards.

  • meta.errors.fail - either true (default) or false to fail the import if there are errors in the meta.xml file. If false, the import will skip extensions with bad metadata.
  • skip.unmodified - either true (default) or false to skip the unmodified records (modification is determined by comparing the dc:modified dates)
  • family / subfamily / tribe / subtribe - used to indicate that an archive is expected to contain records related to a named taxon (only the most specific group is used). If taxon.processing.mode is set to CHECK_TAXA, then the taxon file will be matched against the existing checklist for that group. Taxa which are found in this resource but are outside of the specified group are marked as "Unexpected". Extension records that are not related to taxa belonging to this group will be rejected.
  • taxon.processing.mode - either CHECK_TAXA to check but do not import them, IMPORT_TAXA to import taxa and SKIP_TAXA to skip taxa (useful for organisations which contain a sparse set of taxa such as the families scratchpad)
  • image / key / phylogeny / term / reference / typeAndSpecimen / description / distribution / identifier / measurementOrFact / vernacularName.processing.mode - either e.g. IMPORT_IMAGES to import and delete any records which belong to this organisation but are not found in the archive, IMPORT_IMAGES_DONT_DELETE_GLOBAL to import as before but only remove absent records related to taxa in this subset (e.g. family / subfamily/tribe/subtribe) or SKIP_IMAGES to skip the extension totally.

Identification Keys

  • root.taxon.identifier - The identifier of the root taxon of the harvested key (not required, especially for keys which do not have a root)
  • assume.accepted.matches - Either false (default) or true, to change the way that matching is performed. If this is set to true: When multiple taxa exactly match the supplied string, if only one is Accepted it will be considered a match. If this paremeter is set to false or not set the matcher will fail to match homonyms without authorities.

Phylogenetic Trees

  • input.file.extension - One of nex / nhx / nwk / xml for phylogenies in NEXUS, New Hampshire Extended, Newick or PhyloXML respectively. Useful if the phylogeny is being harvested from TreeBase (and consequently the uri does not have a file extension).
  • root.taxon.identifier - The identifier of the root taxon of the harvested phylogeny (not required)
  • assume.accepted.matches - Either false (default) or true, to change the way that matching is performed. If this is set to true: When multiple taxa exactly match the supplied string, if only one is Accepted it will be considered a match. If this paremeter is set to false or not set the matcher will fail to match homonyms without authorities.
  • phylogeny.title - Set the title of the phylogeny
  • phylogeny.creator - Set the creator of the phylogeny
  • phylogeny.description - Set a description of the phylogeny
  • phylogeny.rights - Set a natural language rights statement for the phylogeny
  • phylogeny.rights.holder - Set the rights holder for the phylogeny
  • phylogeny.license - Set a URI pointing to the license of the phylogeny

IUCN API

The IUCN api resources are harvested from the IUCN web service. The documentation for this webservice is here: https://www.assembla.com/spaces/sis/wiki/Red_List_API

Originally we tried to configure the web service to harvest from each family, but mismatches between the higher classification used by eMonocot and the IUCN Red List meant that some taxa were left out. Instead we decided to configure a single resource which harvests the whole of the monocots (here)

  • family / subfamily / tribe / subtribe - used to indicate that a resource is expected to contain records related to a named taxon. Measurement or Fact records that belong to taxa outside of this group will be rejected.
  • assume.accepted.matches - Either false (default) or true, to change the way that matching is performed. If this is set to true: When multiple taxa exactly match the supplied string, if only one is Accepted it will be considered a match. If this paremeter is set to false or not set the matcher will fail to match homonyms without authorities.
  • license - set the license of measurement or fact records created by this job
  • rights - set the natural language rights statement of measurement or fact records created by this job
  • rightsHolder - set the rights holder of measurement or fact records created by this job
  • accessRights - set the access rights of measurement or fact records created by this job
  • bibliographicCitation - set the bibliographic citation of measurement or fact records created by this job
  • iucnWebsiteUri - The IUCN website URI with ${identifier} as the placeholder for the identifier of the taxon e.g. http://www.iucnredlist.org/details/${identifier}/0

GBIF API

The GBIF api resource harvests documents which contain typeandspecimen records formatted as DarwinCore XML. This data can be produced in two ways:

  1. By using the GBIF RESTFul api (documentation here). This webservice returns lots of information (including rights statements, locality data, collectors codes etc), but is not as performant or reliable as the other method described here. Also, it will only return 1000 records at a time. If there are more than 1000 records the job will attempt to use the startindex parameter to page through the resultset until no more records are returned.
  2. By downloading data from the GBIF data portal (here), extracting the xml file from the archive and putting it somewhere on the network so it can be harvested. This has the advantage that the request will not timeout (a reccurring issue using the GBIF web services) and the number of records returned is limited to 100,000, but less information is provided in the response.

Using the GBIF RESTful API

  1. To use the GBIF RESTFul API, first find the taxon concept key of the root taxon (e.g. family or tribe) you wish to harvest occurrences for - this can be found by searching the GBIF data portal for "Species"
  2.   The integer used in the overview page (i.e. http://data.gbif.org/species/1234) is the taxon concept key.
  3. The uri for the webservice is then http://data.gbif.org/ws/rest/occurrence/list?taxonconceptkey={taxon concept key}&maxresults=1000&{other arguments}&format=darwin&mode=raw&startindex=
  4. taxonconceptkey is the taxon concept key and other arguments are any additional arguments you wish to specify. Useful arguments are coordinatestatus=true to indicate that only records with latitude and longitude should be returned and coordinateissues=false to exclude those which have obvious issues with their coordinates.

Downloading Occurrence Data Manually from GBIF

NOTE: GBIF have a new version of their data portal in UAT at the moment, these instructions refer to the current production version of the GBIF portal. It is possible to do the same thing with the UAT portal but the interface is slightly different (better!)

To Download data manually from GBIF

1. visit http://data.gbif.org/occurrences.

2. Filter the dataset as you wish - typically we set the classification includes filter (to the group we were interested in)

 . . . and also set Coordinate Status is Includes coordinates and Coordinate Issues is No issues detected.

3. Then select Search

5. then Download as Darwin Core (maximum 100,000). This leads to a screen with a throbber whilst the download is being prepared.

6. Once the download is available, click on the link to download the archive (its a zip file). Extract the file from the archive and save it somewhere on the network so that the harvester can access it.

Creating a GBIF Resource in the portal

1. Log in to the portal

2. Go to the GBIF organisation page http://e-monocot.org/organisation/data.gbif.org

3. Click "Create a new resource"

4. Give the resource a unique name and set the type to GBIF

5. EITHER (a) If you are using the GBIF webservices, the URL should be something like http://data.gbif.org/ws/rest/occurrence/list?taxonconceptkey={taxon concept key}&maxresults=1000&coordinatestatus=true&coordinateissues=false&format=darwin&mode=raw&startindex=

OR (b) If you downloaded a file from the GBIF data portal called download.xml and you've put it in the root folder of example.com, the uri which you supply to the portal should be: http://example.com/download.xml? (i.e. with a question mark after the file name) .

6. Click submit

7. Navigate to the resource you've just created

8. If the file covers a single family / subfamily / tribe / subtribe

    a. Click Edit this resource

   b. Go to the bottom of the page and add family / subfamily / tribe / subtribe as a parameter and click Add Parameter

   c. Set the family / subfamily / tribe / subtribe name and click Submit

9. Click Harvest to harvest the occurrence data.

Allowed parameters for a GBIF job

The parameters which can be supplied to the GBIF job are:

  • family / subfamily / tribe / subtribe - used to indicate that a resource is expected to contain records related to a named taxon. Type and Specimen records that belong to taxa outside of this group will be rejected.
  • assume.accepted.matches - Either false (default) or true, to change the way that matching is performed. If this is set to true: When multiple taxa exactly match the supplied string, if only one is Accepted it will be considered a match. If this paremeter is set to false or not set the matcher will fail to match homonyms without authorities.

Users

Q. How can I create a new user account?

A. Users can only be created by registering - select the "Sign in" button at the top of the page, then select "Create a new account". Users should receive an email with a one-time link which they need to select to verify their email address. Only then will their account be enabled.

Q. How can I disable/enable a user account?

A. Login. Go to the users page (www.e-monocot.org/user). Search for the user which you want to disable/enable (or page through the list). Select the user, then click "Edit user". Checking "Account Enabled?" should enable the account.

Groups

Q. How can I create a new group?

A. Login. Go to the groups page. Select 'Create a new group'.

Q. How can I add members to a group?

A. If you are an administrator, once logged in, go to the group you are interested in, insert the new member's email address and click on 'Add a member'.

Q. How can I remove a member from a group?

A. If you are an administrator, once logged in, go to the group you are interested in and select 'Edit this group'. To remove a member simply click on the red button next to the related email address.

Q. How can I add a permission to a group?

A. Permissions cannot be added to groups through the portal user interface.

Q. Which are the permission options?

A. There are Create / Update / Delete permissions for Taxon, Image, Phylogeny, IdentificationKey, TypeAndSpecimen, Place, Term objects, plus Users, Groups, Organisations, Resources, Comments and Annotations, Job Executions and Job Instances. Finally there is the Administrators and a Beta-Tester's role which work more globally.

Scratchpads developed and conceived by (alphabetical): Ed Baker, Katherine Bouton Alice Heaton Dimitris Koureas, Laurence Livermore, Dave Roberts, Simon Rycroft, Ben Scott, Vince Smith