Data Analysis Standards (WP6)
All the deliverables of the work-package have been published, and are made available to the neutron/muon community. This work package is now completed.
- Report on existing software for data analysis (D6.1 Task1) ;
- Report on solutions for developing a common software infrastructure (D6.2 Task2)
- Report on the development of prototype software (D6.3 Task3). The source code can be obtained here
- Report on the evaluation of the prototype software (D6.4 Task 4).
High resolution detectors and higher data rates, among others, make new demands on software that require professional software solutions. In particular, new instruments constitute a considerable challenge for software provision and failure to address this issue leads to a delay in the scientific impact of new investment.
The importance of data analysis has now become clear and is becoming a focal point of efforts to optimise scientific production.
The aim of this networking activity is to determine how to develop, deploy and operate a common data analysis software infrastructure to facilitate joint software development in the neutron scattering community.
Workpackage 6 FP7/NMI3-II project number 283883 tasks (started June 2012):
- Task 1 : Review existing data analysis software and practices of software developers (March 2013)
- Task 2 : Review existing solutions for a common data analysis infrastructure (March 2014)
- Task 3 : Develop prototype software in chosen solution for representative applications (Sept 2014) ; Source code
- Task 4 : Evaluate prototype software (Feb 2015)
Partners: ILL (lead), STFC/ISIS, TUM and JCNS (FRM2), PSI, HZB, CEA, HZG/Hamburg, ESS Lund/Copenhagen
Look at our Meetings here with their minutes.
Tasks and Reports
LiveDVD for testing/installing software (July 2013)
The Data Analysis workpackage and the Computing for Science group at the ILL, provide a ready to run LiveDVD (bootable) which includes a set of neutron scattering software. It runs out of the box, does not affect your current installed system, and can optionally be installed on a hard disk (create an empty partition/area so that the installer use it).
This is especially suited for testing software, schools/workshop/tutorials, and system deployment.
The current Live DVD 'July 2013' is based upon a Ubuntu 12.04 system with the following additional applications:
- LAMP, DAVE, Mantid, Frida, Isaw, nMoldyn, iFit, qtiKWS
- McStas, Vitess, SimRes
- MFit/MView, vTAS, ResTrax
- GRASP, SANSView, SASFit, GenX
- FullProf, PDFgui, F.O.X., OpenMX, GSAS/EXPGUI
as well as:
- C, C++ and Fortran compilers, Java, Python, Perl, Tcl/Tk, MPICH,
- Python libs: Matplotlib, numpy, scientific, tk, numeric
- AbInit, Chemtool, Gelemental, Ghemical, Gperiodic,
- Gromacs, mopac, MPQC, PyMol, Rasmol, Viewmol,
- Avogadro, DrawXtl, CBFlib DX, Fityk, Gabedit, Garlic, Gcrystal,
- Scilab, Octave, PerlDL, PGPLOT, netcdf, ncview, hdf4, hdf5, nexus, hdfview
- GeomView, OpenBabel, ParaView, Maxima, Java3d, Mayavi, qtiplot
- gimp, virtualbox, openoffice, freewrl, partimage, gparted
- March 2011: initial release based on Ubuntu 10.04
- May 2012: release based on Ubuntu 12.04 , adding Frida, Isaw, SANSview, GenX, PDFgui, Vitess. nMoldyn and FreewWRL are buggy.
- June 2012: fixed FreeWRL crash, and nMoldyn. Update your May 2012 installation by installing the following U12.04 amd64 deb packages python-netcdf, python-scientific, libmozjs2d, freewrl).
- July 2013: updated packages, fixed some applications
Neutron-Muon software Debian repository
Software sources: add http://packages.mccode.org/ Debian repository
We now provide an experimental Debian repository with selected data reduction/analysis software. The packages are available for a 64 bits Debian based linux system (e.g. Debian, Ubuntu, Mint, …).
Included packages: bornagain, dave, flatcone, frida, fullprof, genx, grasp, gsas, horace, ifit / reslibcal, isaw, lamp, mcstas-suite, mcxtrace-suite, mfit, neutrons4science, nmoldyn, restrax, root, sasfit, simres, vitess, vtas
To install the packages:
- sudo apt-add-repository 'deb http://packages.mccode.org/debian stable main'
- sudo apt-get update
and then use your package manager (e.g. Aptitude, apt-get, Synaptic, …).
Produced code and software
The following code has been produced:
- Live data reduction server a simple web server which allows to send data files and computational requests to a server, which then dispatches it to computation services, which then report status and results.
- AllToMantid a simple Algorithm that can read/write from any client (e.g. LAMP, iFit, …) and generate Mantid workspaces.
- A set of Mantid loaders and algorithms to be used for ILL instruments (and others), with example data files, scripts, documentation, mainly in Algorithms LoadILL, LoadILLSANS, LoadLLB, LoadSINQFocus, DetectorEfficiencyCorUser, SetupILLD33Reduction