Back from the Global Nodes meeting... a few reflexions

I recently gave a progress report on the development of the NPT at the GBIF global Nodes meeting in Buenos Aires. After the presentation, a long and *very* animated discussion took place between the Nodes representatives, the Advisory Group, members of the GBIF secretariat and myself. A few days later, here are a few reflexions about how the meeting went, which I'd like to share with the community.

1. The NPT project triggers a lot of interest: feedback from the Nodes representatives during the meeting was intense and precious for the coordination of the project;
2. The communication needs to be improved: apparently, several participants weren't well informed about the NPT project. Additional communication channels will be explored. It should be stressed that a lot of efforts so far have been devoted to technical aspects as well as requirements capture and their transfer to software documentation.
3. There is a risk of digital divide between the participants. A special effort should be devoted to limiting this divide, by adapting the development strategy. The possibility for early training should be explored as well;
4. The governance and responsibilities of the different bodies managing the project, as well as the role of the project coordinator need to be clarified. The decision making process didn't seem to be transparent enough;

As a followup, I'm intending to take the following actions:
1. setup a shopwindow website (in additions to the existing websites) giving the basic information (background, functionalities, roadmap, governance) about the project.
2. prepare an information paper, to be circulated widely within and beyond the community, possibly in several languages.
3. setup a "dummy" website to give a general idea of what the first version of NPT is going to look like
4. setup face to face meetings to the best extent feasible to address issue (3).
5. prepare a general survey to find-out about the potential level of pickup of the first version by the community (how many instances would be installed from NPT1)
6. prepare a survey on the technical capacity of the network


Your comments and feedback are most welcome on this.

Updated Mindmap available

If you want to get detailed information on the NPT, in an interactive, dynamic way, you can view/download the new Mindmap, available from the x-mind share space. The map allows you to drill down in the details of the project, and visualize the many facets of the NPT. Stay tuned, as this mindmap is often updated. You can also re-use/re-publish the map yourself, and share with the ret of the community!

From Problems to Solutions

We've just held our first NPT technical meeting in Brussels. After a lot of care was devoted to capturing and analyzing what the potential users of the NPT would want (Scoping and Requirements) and how the system should behave in many different circumstances (Software Requirement Specification), we were finally able to go into the solutions during a focused workshop recently organized. Considering the extent of the GBIF network, the extreme heterogeneity of the wishes and needs of the community, the fast pace of technological developments and the time constraints, we found that applying a one-size-fits-all approach will simply not work. Nevertheless, the NPT will provide a custom solution which will be fit for many situations.

The NPT Technical Task Force (TTF) agreed:
• To have a phased approach for implementing the NPT, provisionally referred to as NPT1 and NPT2
• That both implementations will be based on the Drupal framework which supports the development of custom community-driven modules
• That both NPT1 and NPT2 will interact with the GBIF registry and the GBIF Data Portal via web services
• That both NPT1 and NPT2 will retrieve GBIF content through the GBIF Data Portal

Briefly, NPT1 will be just focusing on basic need of national nodes that have no IT capacity at all, which will have a set of functionalities allowing nodes to obtain web presence. NPT1 will be introducing the necessary modules to interact with GBIF web services, which will be further used in NPT2.
Encapsulating the modules and profiles of NPT1, NPT2 will introduce a sophisticated geospatial component which will allow nodes to process, manage and visualise biodiversity data directly within the NPT.

An information document will soon be circulated as part of the information package for the NPT session of the Nodes Global meeting, as well as a detailed report for the TTF meeting. Both documents will be posted on the NPT Google Code site and on the NPT GBIF community site.

We're also on Facebook

Join our Facebook page Do you want to share biodiversity information? The NPT has the power to get things moving! join our Facebook group, and feel free to join us!

A few words about the SRS document

A Nodes Portal Toolkit (NPT) Software Requirements Specification (SRS) document is being prepared as the next logical step after the NPT Scoping and Requirements (S&R) document, which was prepared during the NPT Advisory Group workshop, held in Madrid in March 2011 and publicly released in June 2011. The purpose of the SRS is to describe the software requirements for the NPT, reaching a level of detail which makes it fit to be circulated amongst the NPT stakeholder, and in particular amongst the potential developers, as well as to prepare a high-level document focusing on the architectural and design aspect of the NPT. The SRS focuses on functional characteristics of the NPT. Data structures and user interfaces will require further specification and development using an iterative, prototype-oriented software development methodology. The purpose of the current version of the SRS document is to clarify, validate and verify the requirements, focusing on the functional aspects. The SRS captures and describes the desired behavior of the end product. The SRS document is intended to be a planning instrument for the future development of the NPT and its potential derivatives.

NPT Technical meeting coming up

A meeting will be convened at the end of this week to delineate some technical aspects of the Nodes Portal Toolkit. 14 participants will be attending the meeting, organized in Brussels and will tackle the high-level Architecture aspects, as well as the development plan for this (extremely) challenging project, after a reality check of the Toolkit's software specifications. This technical meeting will be a first step towards the development phase of this community-driven project. Stay tuned!

What are we up to?

Wondering what's going on with the NPT project?
The easiest way to find out it to take a look at our Roadmap! It gives our followers an overview of the main milestones and where we are at the moment.
Right now, advanced technical documentation is in preparation, following up the release of the NPT Scoping and Requirements document, to ensure the development phase rolls and that minimal space is left for questioning and doubts for the developers. This demands in-depth analysis (reality checks) of the available tools, standards, data, modules, components, other technical and human contraints, etc... and will shape the overall design of the NPT.
The next outcome is an NPT SRS (Software Requirements Specification) document, detailing interface requirements and system features, which will be followed by a Software Design Document.
At the same time, discussion are ongoing to reconcile a team of developers that will get their hands in the grease and make it happen.
If you have any questions or comments, you can contact the NPT coordinator directly, or use this Blog, or the Google Code website to inject your feedback (which is most welcome!).

NPT Mindmaps

An interactive mindmap is available for all to visualize a series of components of the future NPT. The mindmap mainly focuses on the requirements, and front/backend functionalities of the Toolkit. As usual, comments and feedback are more than welcome!

Requirement 15: Exploration of APIs

Availability of APIs and Web Services. The design and creation of Application Programming Interfaces (APIs) that expose NPT functions should be explored to allow third parties to use NPT functionalities. Web services should also be explored.

Requirement 14: Sustainability

Sustainability embedded at each stage of development.  The developers of the NPT should be mindful of the strategic implications of development choices to the long term sustainability of the NPT in a number of key areas: underlying hardware and software; development tools and frameworks; level of effort for system administrators and portal managers; level of interest and engagement of users, including researchers, data providers and other interested visitors.  

Requirement 13: Simplicity

Simple to maintain. Maintenance in terms of user interface configuration, user rights management, and data source(s) customisation should be done at the management level with visual tools. The installation, operation and maintenance at the system administrator level should be well documented and administrator manuals and guidelines should be carefully provided. 

Requirement 12: Discoverability

Discoverable through search engines. The NPT should have appropriate html metadata tags that allow the portal itself to be discovered through search engines. Those metatags should be populated with metadata that allow portals to be distinguished from each other, for example as they deal with distinct themes, regions, or countries. Metadata made available to search engines should aim primarily at making the data easily discoverable through e.g. Google. 

Requirement 11: Tracability

Capability to efficiently track portal usage and database content statistics. Both end-users and portal managers should be able to view statistics about the portal usage. Statistics can include number of hits, number of occurrences, number of new records monthly, number of queries, number of downloads, total number of downloaded records. In the case of data downloads, there should be brief metrics collected on how the data are used. 

Requirement 10: Community-driven

Community-driven. By including opportunities for user feedback, the system should evolve by iterations and respond to user (NPT manager and end-users) needs. User interactions can also happen by using stored queries. The implication of the community is a core requirement for the NPT, and the objective is to ensure that the tool has enough buy-in for it to be allowed to evolve in the future.

Requirement 9: Collaboration

Open collaborative development. The development of the application should be consistent with open source, allowing the core code and any modules developed to be improved over time. 

Requirement 8: Documented

Documentation and code maintenance. A clear and standard practice for the maintenance and documentation of code should be set in place, agreed and communicated as a best practice.  Any features developed outside the core of the NPT should follow that practice. Environments such as Google code should be used to support this collaboration. Where needed, other environments can be adopted to ensure all players have access. Regarding non visual tools, procedures and routines for installation, configuration and maintenance can be accepted if they are well documented and guidelines provided all the way down to the simpler operation. Upgrades should be made as simple as possible: the possibility to have them ran automatically should be examined.

Requirement 8: Friendliness

Intuitive and friendly end-user interface. The interface for end-users and to a lesser extent portal managers should be relatively intuitive and easy to use, without requiring extensive technical assistance, support or off line user guides. For portal managers, such guides may be needed to explore advanced functionalities, but those advanced functionalities should be as intuitive as possible.  

Requirement 7: Configurability

Configurable. The NPT should be configurable in terms of features, data sets, modules and be able to plug in various layout designs and on different CMS (Content Management Systems). In the beginning, the NPT should at least provide support for popular CMS, such as Drupal and/or Zope. It should include provisions to add other CMS support at a later stage of development.

Checkout the NPT mockups

Want to have an idea as to how the NPT user interface will look like? Take a sneak peak at the NPT mockups here!
Don't hesitate to comment, or send us your ideas!

Requirement 5: Extensibility

Extensible. The NPT development and implementation should follow design principles that will enable future growth, both in terms of the addition of new functionality or through modification of existing functionality. In addition, NPT developers should provide the possibility to consume services and functions that are created elsewhere. Examples might include taxonomic data sources or mapping functions or web map services. 

Requirement 6: Multilingual

Multilingual. The user interface at the front end should support multilingualism. At the backend, the primary interface language should be English. 

Requirement 4: Robustness

Robust. The NPT should be thoroughly tested to ensure robustness on three fronts: (1) the NPT should be able to cope (without crashing) with a wide range of stress situations, and it should perform reasonably under important workload (queries and data discovery). User experience, both front end and back end, should be as intuitive and friendly as possible; any errors should be trapped and not seen by users, but systematically addressed as priority requirements or issues; (2) it should also allow the management of dependencies on external services (for example, external data calls). 

Meet the NPT GoogleCode site

For those of you following the developments of the NPT project, a googlecode site is available. The website includes documentation, downloads, resources, a wiki, issue tracking and information about the NPT. In the near future, the site will give access to source code.

A concept for building the content of the NPT

NPT meets ViBRANT

Burke and Bruno recently met with the ViBRANT team (Vincent Smith and Simon Rycroft) at the NHM, in London. With GBIF’s involvement in ViBRANT project and NPT as a part of Work Package 8, the purpose of this trip was to discuss with ViBRANT team about the possible synergies between the ViBRANT developments (in the continuity of the Scratchpads project) and some of the components of the NPT. The idea was to explore the potential of certain Drupal components to serve as a basis for the NPT. It was found that most of the backend requirements identified by the Advisory Group were already available from the core Drupal components. It was also found that some Drupal modules seem to be fit for our use for the NPT. This approach might be fit for the development of the NPT, especially given the technical competences of some core developers, and the proof of concept of using a combined framework (Drupal + OpenGeo), as adopted for the new iOBIS (Ocean Biogeographic Information System) webportal.
The ViBRANT team has agreed to take part in the developments of the NPT.

Requirement 3: Compliance with GBIF informatics

Compliant with the GBIF Informatics Infrastructure / Architecture. To ensure the interoperability of the NPT with other GBIF tools such as HIT and IPT, NPT should be developed in a manner that is compliant with GBIF Informatics Infrastructure and Architecture. The interoperability of the NPT with other GBIF informatics tools should be aiming at optimizing current development, and ensuring the same standards are adopted and that resulting data flows are kept simple and consistent (i.e. not affecting the data quality of the data system as a whole).  When other informatics tools impose constraints on NPT functionality, NPT developers should communicate those constraints and work in concert with GBIF informatics staff. 

NPT: the next steps

The NPT is still in the main design/architecture phase. The first version of the NPT scoping and requirements document will be released soon, and posted on the NPT Google Code site. 
Tomorrow, the NPT coordinator will meet with the ViBRANT crew in London together with two GBIFS representatives to examine common solutions to the respective developments involved in the NPT and the ViBRANT project. The idea is to optimize the work and make sure systems will be able to "talk to each other". 
The next steps include:

Preparation of an Architecture document (aimed at developers)

Finalization of wireframe and Mindmap

Informal pre-anouncement for DEVs 

Call for expressions of interest by NPT DEVs prepared, circulated and issued 

Evaluation and selection of NPT development partners

NPT coding camp I

Stay tuned!

Requirement 2: multi-platform

Multi-platform. The NPT should be developed so that it can be hosted on Windows, MacOS or Linux operating systems from the beginning. This will allow users (NPT administrators) to work with the OS they are most familiar with, including free and open source solutions.

Methodology: the FURPS+ System for Classifying Requirements

An interesting post by Peter Eeles, Senior IT Architect, IBM on Capturing Architectural Requirements. The article suggests a systematic approach to capturing architectural requirements to ensure that these elusive, and yet extremely important, system specifications are not overlooked.
See full article here.

Requirement 1: Free & Open source solutions

To the best extent possible, the system should be based on layers of open source solutions such as PHP, MySQL, Apache. Open source application development frameworks or libraries such as JQuery (javascript), CakePHP, that streamline and simplify both code development and updates should also be used. Web standards (HTML5, CSS3, JQuery) should also be considered. Easy maintenance of the application is an important criteria.

What you need to know about the NPT

The Nodes Portal Toolkit (NPT) is envisioned as a suite of informatics tools designed to help GBIF Participants to deploy, maintain and extend biodiversity data portals (at the national, regional or thematic levels). The GBIF Participant Node Managers Committee recommended in October 2009 to fully engage the nodes community in the development cycle of the NPT, especially to leverage the technical capacity available across the GBIF network.
Following this recommendation a technical advisory group with representatives from the Nodes Community was convened, and a workshop of this group held in Montreal, Canada, in December 2009 . This group provided specific guidance on 1) the overall scope for the NPT, 2) structure and components, and 3) development approach.
This group also recommended hiring an expert from the community to coordinate the NPT development (especially to serve as a liaison between the NPT end users, developers, strategic partners, the GBIF Secretariat, and the Nodes Committee, among others). The call for this position was issued in August 2010 , and a NPT Project Coordinator selected and appointed in October 2010 .
The NPT Project Coordinator setup an NPT Advisory Group workshop, which was held in Madrid in March 2011. It is an attempt to scope and capture the requirements of the NPT. The meeting was held in an interactive way and this document was drafted collaboratively by all participants to this meeting.
The Advisory Group is composed of the following persons: Cees Hof (GBIF-NL), David Remsen (GBIF-S), Douglas Hyde (NatureServe Canada), Francisco Pando (GBIF-ES), Frank Oguya (GBIF-KE), Guy Baillargeon (GBIF-CA), Maria Auxiliadora Mora (GBIF-CR), Mélianie Raymond (GBIF-S), Mihail-Constantin Carausu (GBIF-DK), Nabil Youdjou (GBIF-AQ), Nicolas Noé (GBIF-BE), Peter Desmet (Canadensys), Steve Wilkinson (GBIF-UK), Tim Robertson (GBIF-S), Wouter Addink (ETi). The NPT Project Coordinator (Bruno Danis, GBIF-AQ) and the GBIF Senior Programme Officer for Nodes (Juan Bello, GBIF-S) also attended the workshop.

A few words on the NPT Advisory Group workshop

The first NPT-AG workshop was hosted by Francisco Pando, GBIF-ES, at the Real Jardin Botanico, Madrid, SPAIN, 4-6 March 2011.
The workshop was intended to be as participative as possible, with constant feedback from participants, and using collaborative reporting and brainstorming tools, such as Google Documents, Mindmaps, Mockups. This approach helped delineating the needs for the NPT and identify priorities, resulting in a set of realistic functionalities for the NPT.
The meeting ended with a final sprint, during which the NPT-AG drafted a first version of the NPT scoping and functionalities document. This document will soon be circulated via the NPT Google Code site.

New NPT blog is up

This blog is devoted to stream information about the development of the GBIF Nodes Portal Toolkit (NPT). It will keep you informed about various aspect sof this development, as a complimentary channel to the NPT Google Code site which is already up and running and offers a more structured access to information pertaining to the NPT.
Stay tuned for more...