The next EDNA full meeting will take place at the ESRF on September 23rd. It will be preceded by a mxCuBE meeting and succeeded by an ISPyB meeting. For more information and registration to the EDNA meeting (and also the mxCuBE/ISPyB meetings) see:
http://www.esrf.fr/events/conferences/mxcube_edna_ispyb_meetings
Monday, August 30, 2010
Thursday, April 22, 2010
EDNA Dev Kernel code camp at DLS 26-27th April
Diamond will be hosting a virtual code camp on April 26th and 27th with the aim to add some kernel upgrades and generally improve the documentation.
The areas which will be covered in detail are as follows, they are based on CCP4's first use of EDNA and also comments from DLS from the use of MXv1. This is not an exhaustive list, and any other kernel issues which could do with addressing will be welcomed and added before the meeting.
The areas which will be covered in detail are as follows, they are based on CCP4's first use of EDNA and also comments from DLS from the use of MXv1. This is not an exhaustive list, and any other kernel issues which could do with addressing will be welcomed and added before the meeting.
- Documentation
- Automatic unpacking of input objects
- Definition of methods inside plugins
- Allow use of state in plugins
- Define plugin types in data model
- Should input XML be validated
- EDNA run folder location (Possibly an MXv1 issue, but may run deeper)
The Agenda to try to address these issues is as follows (Times are UK) :
Monday
- 09:00 - Coffee + Setup Maratech etc.
- 09:30 - Discuss Kernel changes as stated above, and divide the documentation writing tasks as Olofs plan http://www.edna-site.org/joomla/index.php?option=com_wrapper&view=wrapper&Itemid=6 Should include a code review of some DIMPLE plugins as examples of where issues came from.
- 12:00 - Lunch
- 13:00 - Open session for kernel development
- 4:00 - Code Reviews
- 14:30 - Coffee
- 15:00 - Documentathon - try to get as much doc in place in 30 mins.
- 15:30 - Open Session for kernel development
- 16:30 - Code Reviews
- 17:00 - Close
Tuesday
- 09:00 - Plugin Development (i.e. try to use the new kernel changes in your own code, also the new docs)
- 10:00 - Documentathon - try to fill in any missed bits of doc
- 10:30 - Coffee
- 11:00 - Discussion over kernel changes made, and there implementations.
- 12:00 - Lunch
- 13:00 - Final Session for Kernel Developments
- 14:30 - Code reviews
- 15:00 - Coffee
- 15:30 - Closing discussions etc plus workshop outcomes.
The whole thing will be done using Maratech, and is really aimed for seasoned Python programmers who have written EDNA plugins and are happy to help with making kernel changes.
Friday, January 29, 2010
EDNA Bugzilla server now open for anonymous read-only access
As of today the EDNA bugzilla server is open for anonymous read-only access. Note that you still have to have an account for creating new bugs and posting messages on the Bugzilla server.
Friday, December 18, 2009
Developers workshop/code camp 18 January 2010 DLS
Sorry for the short notice but we would like to announce a plugin developers meeting for EDNA on the 18-20th of January at Diamond.
The schedule of the workshop will be more of an introductory level for new developers on the Monday then more of a code camp style where tasks for kernel developments and specific plugin developments (e.g. MXv1 and V2) will be undertaken in breakout groups for the remainder of the time.
We anticipate that the workshop will be very popular so expression of interest/commitment to attend early is recommended! We will try and organise accommodation at http://www.stfc.ac.uk/About/Conts/Find/Coseners/Introduction.aspx though all travel and accommodation expenses will be the responsibility of the attendee and their home institution.
Early expressions of interest would be very much appreciated! I hope you haven't all gone away for Christmas already, if you have, I hope your having a good time!
Alun
The schedule of the workshop will be more of an introductory level for new developers on the Monday then more of a code camp style where tasks for kernel developments and specific plugin developments (e.g. MXv1 and V2) will be undertaken in breakout groups for the remainder of the time.
We anticipate that the workshop will be very popular so expression of interest/commitment to attend early is recommended! We will try and organise accommodation at http://www.stfc.ac.uk/About/Conts/Find/Coseners/Introduction.aspx though all travel and accommodation expenses will be the responsibility of the attendee and their home institution.
Early expressions of interest would be very much appreciated! I hope you haven't all gone away for Christmas already, if you have, I hope your having a good time!
Alun
Tuesday, December 1, 2009
EDNA in Java
The idea of being able to call EDNA from within java has been discussed many times, and Olof has recently produced an eclipse plug-in which can make these calls. However the way this is done is by using the command line based launchers for EDNA, and hence there is some overhead involved.
The work that I have been doing extends this idea to using java embedded python (jepp) to make the calls to the EDNA plugins directly. This allows for the plugins to be called independently without the general overhead of the framework, and also removes the need to start up a separate shell for each plug-in.
Jep has some issues with being able to pass information between java and python, however as EDNA relies on XML to pass information between different plugins, Jep can be used quite effectively.
Initial work on this topic has now been completed, and this is now working well in a test environment. The next step is to create a full API which allows the Execution of various plugins easily in the Java environment.
However there were a few points which are worth pointing out, as they took a long time to fully get working.
1. Environment variables, It is important to set both the environment variables stated, if only the LD_LIBRARY_PATH is set, most things work completely fine however there are very obscure errors with imports which take a while to track down, a small typo in the LD_PRELOAD path can cause a lot of trouble. So, if there are any issues with import statements such as "import random" this is where to check first.
2. The biggest problem is with imports, if a problem occurs in an import, then jepp will always report the error as being that the first import that was called from the jepp environment cannot be found, even if the import that failed was way down a long list of dependencies, and the error was actually something completely different. Unfortunately there seems to be no way of retrieving the actual exception, either through Python or java, so you’re down to lots of print statements to find where the problem is.
3. The last issue which caused a lot of problems, especially as it was used in a class definition of one particular import, so the error was swallowed, is that as jepp is running on the embedded platform there is no sys.argv. This means that PySys.argv also doesn't exist. This however can be worked round by as one of the first things we do in Jep is to set sys.argv to basically a sensible set of values. This just needs to be a list of strings.
Basically that's then it. It’s then quite easy to call the plugins to be run, and give it an xml string input. And the main thing is that no changes are required in the EDNA code itself. This should be easily combined with the work that Olof has done to make it very easy to use the plugins. Plus the speed of execution is very high in comparison to launching the process externally. More updates to follow, when the full API is finished, and properly integrated into the Eclipse EDNA plugin.
The work that I have been doing extends this idea to using java embedded python (jepp) to make the calls to the EDNA plugins directly. This allows for the plugins to be called independently without the general overhead of the framework, and also removes the need to start up a separate shell for each plug-in.
Jep has some issues with being able to pass information between java and python, however as EDNA relies on XML to pass information between different plugins, Jep can be used quite effectively.
Initial work on this topic has now been completed, and this is now working well in a test environment. The next step is to create a full API which allows the Execution of various plugins easily in the Java environment.
However there were a few points which are worth pointing out, as they took a long time to fully get working.
1. Environment variables, It is important to set both the environment variables stated, if only the LD_LIBRARY_PATH is set, most things work completely fine however there are very obscure errors with imports which take a while to track down, a small typo in the LD_PRELOAD path can cause a lot of trouble. So, if there are any issues with import statements such as "import random" this is where to check first.
2. The biggest problem is with imports, if a problem occurs in an import, then jepp will always report the error as being that the first import that was called from the jepp environment cannot be found, even if the import that failed was way down a long list of dependencies, and the error was actually something completely different. Unfortunately there seems to be no way of retrieving the actual exception, either through Python or java, so you’re down to lots of print statements to find where the problem is.
3. The last issue which caused a lot of problems, especially as it was used in a class definition of one particular import, so the error was swallowed, is that as jepp is running on the embedded platform there is no sys.argv. This means that PySys.argv also doesn't exist. This however can be worked round by as one of the first things we do in Jep is to set sys.argv to basically a sensible set of values. This just needs to be a list of strings.
Basically that's then it. It’s then quite easy to call the plugins to be run, and give it an xml string input. And the main thing is that no changes are required in the EDNA code itself. This should be easily combined with the work that Olof has done to make it very easy to use the plugins. Plus the speed of execution is very high in comparison to launching the process externally. More updates to follow, when the full API is finished, and properly integrated into the Eclipse EDNA plugin.
Friday, November 20, 2009
EDNA logo
The EDNA project has not yet any logo. The one used until recently on the EDNA Wiki was rapidly put together and meant to be a temporary logo - which has remained in its place for more than two years...
In the last developers' meeting in Grenoble I suggested a new logo in the meeting introduction :
The idea behind this logo is the following picture, which was presented for the first time in the EDNA developers' meeting at the DLS in February this year:
This is the EDNA project in a nutshell : We are trying to build a framework and applications that are modular, that share data models, implements workflows and are robust thanks to the testing framework. The base of the EDNA project / collaboration is the project management with the executive committee, project manager, project agreement, coding conventions, code reviews etc.
The proposed logo is not yet approved, so if you have any comments please add them to the blog or send them to me!
In the last developers' meeting in Grenoble I suggested a new logo in the meeting introduction :
The idea behind this logo is the following picture, which was presented for the first time in the EDNA developers' meeting at the DLS in February this year:
This is the EDNA project in a nutshell : We are trying to build a framework and applications that are modular, that share data models, implements workflows and are robust thanks to the testing framework. The base of the EDNA project / collaboration is the project management with the executive committee, project manager, project agreement, coding conventions, code reviews etc.The proposed logo is not yet approved, so if you have any comments please add them to the blog or send them to me!
Estimation of sensitivity to radiation damage by sacrificing a crystal
The goal of this new development is to enable the estimation of radiation damage by sacrificing a crystal. The proposal of Sasha Popov and Ricardo Leal is to this it in four steps:
1. Collect one or several reference images from the crystal to be sacrificed.
2. Use EDNA characterisation and a new version of BEST (3.3) to calculate a strategy that both "fries" the crystal and collects some reference frames at regular intervals. The proposal is that this strategy consists of 10 data collection sets, where each data set consists of:
- A couple of reference frames collected at e.g. 5 % transmission
- A data set for burning the crystal with 100 % transmission, the idea is to give a certain dose to the crystal. The images of this data set are not needed in the analysis.
3. Collect this strategy using the BCM (e.g. mxCuBE)
4. Analyse the reference frames with new EDNA plugins. What I understand being the most important parameter to monitor is the increase of B-factor with dose/exposure time. The angle of the slope of this line is a measure of the crystal life time on the beamline.
I will work together with Ricardo Leal to implement step 2. The work in EDNA for step 2 should be straightforward:
- Modification of the MXv1 and BESTv1_1 data model, add "strategyOption" [XSDataString] to XSDataDiffractionPlan (bug #379).
- Update the BEST plugin to version 1.2 in order to use the strategyOption attribute for running BEST with the correct command line option (bug #384).
This new attribute obsoletes the current attribute "anomalousData" [XSDataBoolean] since the anomalousData could be passed as a string to "strategyOption". I suggest though to keep "anomalousData" in the data models in order to not break any existing plugins.
Step 4 will be developed later...
1. Collect one or several reference images from the crystal to be sacrificed.
2. Use EDNA characterisation and a new version of BEST (3.3) to calculate a strategy that both "fries" the crystal and collects some reference frames at regular intervals. The proposal is that this strategy consists of 10 data collection sets, where each data set consists of:
- A couple of reference frames collected at e.g. 5 % transmission
- A data set for burning the crystal with 100 % transmission, the idea is to give a certain dose to the crystal. The images of this data set are not needed in the analysis.
3. Collect this strategy using the BCM (e.g. mxCuBE)
4. Analyse the reference frames with new EDNA plugins. What I understand being the most important parameter to monitor is the increase of B-factor with dose/exposure time. The angle of the slope of this line is a measure of the crystal life time on the beamline.
I will work together with Ricardo Leal to implement step 2. The work in EDNA for step 2 should be straightforward:
- Modification of the MXv1 and BESTv1_1 data model, add "strategyOption" [XSDataString] to XSDataDiffractionPlan (bug #379).
- Update the BEST plugin to version 1.2 in order to use the strategyOption attribute for running BEST with the correct command line option (bug #384).
This new attribute obsoletes the current attribute "anomalousData" [XSDataBoolean] since the anomalousData could be passed as a string to "strategyOption". I suggest though to keep "anomalousData" in the data models in order to not break any existing plugins.
Step 4 will be developed later...
Subscribe to:
Posts (Atom)
Posts
