WorkUnits

From FaHWiki
Revision as of 07:22, 25 September 2010 by MtM (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

General

The projects available for processing are distributed to the Folding@Home clients in Work Units (WU).


A WU is a collection of files, located in a work directory created by the client, with data about atoms, bonds, temperature and likely even more parameters.
See also: FAH files and Folding-community: dlucent's post in Folding@home, *** NOW WITH RATE EQUATIONS!!! ***


Trajectories

Each project consists of multiple WUs, and each WU calculates a slightly different portion of a trajectory for a particular protein. These trajectory parts are identified by the Run, Clone and Generation numbers which can be found in the file File icon.gif FAHlog.txt.

Project: 2401 (Run 40, Clone 21, Gen 1)

Because of the various trajectories for each particular project, the completion time of each WU may differ somewhat depending on the trajectory of the project it calculates. Within a single project, the variations are generally quite small; variations between projects are generally large.


The name and project number of a particular WU does not tell you the specifics about that WU. Thus when the GUI client, or any other monitoring tools say you are processing a particular project, you are in fact processing a small portion of that project. If you get two WUs from the same project, you will be calculating different portions of that project.


WU Types

WUs come in all shapes and sizes. This diversity is caused by the different proteins being examined, the various stages of that examination and the methods used to do the examination. These differences can be "felt" by the amount of disk space a WU requires for storage, the memory it uses while operating and the CPU time it takes to complete. Typically a project is designed to take 2-4 days on the benchmark machine to complete. Meaning it is worth 220-440 points.
See also: WU Credits


In addition to normal WUS, there are four other types of WUs available to Folding@Home participants:


Deadlineless Work Units

The WUs for all the projects have deadlines, with the exception of a small selection of projects whose WUs go without deadlines. Deadlineless WUs are available to clients that have been configured to receive WUs without deadlines. This type of WU is meant for clients who lack the speed to complete normal WUs within deadlines and those who originally ran Genome@Home on slow hardware.

Deadlineless WUs can also be cached in sets of 10 WUs.

Note: The caching of WUs is not intended for clients with the capacity to complete normal WUs within deadlines. All normal projects are designed to be completed within their deadlines by machines of reasonable performance, even if they are on non-permanent Internet connections like dial-up.


Use deadlineless WUs only on machines which cannot complete normal WUs within deadlines! And don't cache WUs unless really necessary, for instance in a Sneakernetting scenario.


Deadlineless Work Units Unavailable

Deadlineless Work Units are no longer available. And there are currently no plans to reinstate their use:

Deadline-less work units. We don't have any current plans to reinstate these,
but we can't rule them out in the future. These existed in the past both to
accommodate slow machines and because there were some results that we could
get back very slowly and still do useful science.

Folding-community: kasson in "SMP for Windows Beta - Just thinking outloud"


Large Work Units

Large WUs, also known as Big WUs (or BigWUs), are available to clients configured to allow the receipt and transmission of WUs larger than 5 MB. These WUs typically require more system memory, but are defined by the larger download and upload sizes, and not all Big WUs use extra memory. The current bonus structure (at the discretion of Pande Group) makes Big WUs worth double the credit of normal CPU client WUs.


To enable this feature, answer yes to this client configuration question: Allow receipt of work assignments and return of work results greater than 5MB in size (such work units may have large memory demands (no/yes) [no]? Console client configuration options?


Enabling Big WUs increases the Packet Size Limit from 5241856 bytes (5 MB) to 524286976 bytes (500 MB).

Extra-Large Work Units

This is public trial of a new category of WUs. It is also known as bigadv WUs. It requires powerful hardware and can run only on SMP2 Client. Their requirements are higher than Large Work Units. More details can be viewed in this thread


Advanced Work Units

The New Advanced WUs are available to clients who are started with the -advmethods parameter. These WUs are in the final stage of testing before being made available to all other Folding@Home participants. Please note that there is an increased chance for errors and other problems with these WUs.


Beta Work Units

Beta WUs are only available to members of the Folding@Home Beta Team. There is a higher chance of problems and errors with these WUs.


WU Assignment

A WU is assigned to a client by the Folding@Home Assignment Servers. You cannot choose which WU you want to receive from those servers as the servers always decide which WU they will issue to whom.


It is possible to set a few Work Unit preferences in the F@H client, but it is not guaranteed that these preferences will be adhered to. You can, for instance, configure your client to receive WUs without deadlines. This preference is almost always adhered to, because this type of WU is intended for clients who lack the speed to complete normal Work Units within their deadlines, but there is no guarantee. You can also configure your client to receive Large Work Units. This tells the Assignment Servers that it is OK to issue larger than normal WUs to your client. This type of WU may place much higher memory demands on your system. Finally, you can use the -advmethods flag to set your client to be available for trying out new advanced simulations before they are released to other Folding@Home participants. Please note that because these WUs are in the final stage of testing, there is an increased chance for errors and other problems.


A WU representing a unique trajectory within a project is generally not distributed to multiple clients. Only when a WU is not returned by the preferred deadline will it be reissued to another client. If the reissued WU is returned by the original client before the final deadline, that client will still be awarded the credit. When the client who received the WU when it was reissued also returns the WU before the final deadline the results of both clients will be compared.


A WU will also be reissued if the final deadline is exceeded, but the client who exceeded the deadline will not be awarded credit. Also when a WU terminates unexpectedly, for instances with an Early Unit End, it will it be reissued to see if the termination occurs on more clients.


WU Processing

The WUs are not processed directly by the Folding@Home client itself. Instead the client executes specialized programs called Cores. Each core consists of code which is optimized for a particular type of processing. Some of the cores used by the Folding@Home client are Tinker, Gromacs, and QMD.


The F@H client handles the up- and downloading of WUs, and the cores do the actual work. The cores are started with a number of commandline parameters by the Folding@Home client. The parameters tell the core which directory and queue index to use, the Process ID (PID), the version of the F@H client, etc.


Checkpoints

By default all cores, except the Tinker core, will save intermediate checkpoint files of the current WU's progress. When the client is stopped and later restarted, it will resume work starting from the latest checkpoint. Improper shutdown of the client can corrupt the checkpoint file whereby its checksum will no long match; this will cause the client to restart processing from the beginning of the WU.


By default a checkpoint is written every 15 minutes, but the client can be configured to use a different interval. The interval in minutes is passed to the cores with the -checkpoint commandline parameter.


When a checkpoint is triggered, it will be logged in the FAHlog.txt file. Depending on which core is running the current WU, the logmessage will be one of the following:

Timered checkpoint triggered 
Writing checkpoint files 

The logmessage indicating that the client has restarted from a saved checkpoint will be one of the following:

Restarting from checkpointed files
Starting from checkpoint

The checkpoint file is located in the F@H client Folder2.png work directory as:

wudata_<XX>.chk 

where "<XX>" is the two digit number of the queue index of the current WU.
See also: FAH files


WU Deadlines

All Work Units are time sensitive and have deadlines. There are 2 deadlines attached to most WUs:

  1. Preferred deadline
  2. Final deadline


When a client does not return a completed WU within its Preferred deadline, the WU is assumed lost. The WU will be reissued to another client to make sure the results will be returned. Every WU is important to each project even though a WU is only a small part of a single project, and part of the Folding@home Project. Below is the relationship between the WU assigned to you and its deadlines:


Classic Client WUs:

Before Preferred Deadline - You will get assigned Credit

Exceed Preferred Deadline - WU will be reissued. You will get assigned Credit

Exceed Final Deadline - WU is useless. You won't get any Credit


GPU2 Client WUs:

Before Preferred Deadline - You will get assigned Credit

Exceed Preferred Deadline - WU will be reissued. You will get assigned Credit

Exceed Final Deadline - WU is useless. You won't get any Credit


GPU3 BETA Client WUs:

Before Preferred Deadline - You will get assigned Credit

Exceed Preferred Deadline - WU will be reissued. You will get assigned Credit

Exceed Final Deadline - WU is useless. You won't get any Credit


SMP2 BETA Client WUs: (normal and bigadv)

Before Preferred Deadline - You will get Bonus Credit (The Bonus varies from system to system and increases for faster completion and return)

Exceed Preferred Deadline - WU will be reissued. You will get Base Credit (It is much less when compared to Bonus Credits)

Exceed Final Deadline - WU is useless. You won't get any Credit


Note: Credit is granted for completed WU that has been uploaded to the Servers successfully. Credit is often given to some WUs which cannot be completed because of an error. In both cases, they will not appear in the official stats pages until at least an hour or more after the results are uploaded.


See also: How Important are Deadlines? and How are Deadlines set?


WU Credits

All the WUs for a specific project are worth the same amount of credits, but projects differ in the amount of credit a WU for that project is worth. The credit a project is worth may change over time (although that is fairly rare), for instance when it turns out that a project requires much more time to complete than was initially determined.


Stanford hosts a webpage with all the active projects. Among the other information on that page is the credit each project is worth at that time. You can also use one of the many Third Party Contributions which can display your current WU and its credit, or the whole credit history for the projects.


Determining Credit

The amount of credits awarded for completed WUs for each project is based on a benchmark applied to all the projects.


Benchmarks are performed on a 2.8 GHz P4 running Linux with SSE2 disabled. The machine is set to receive 110 Points Per Day. This means that a project worth 110 points should be completed in 24 hours when running on an identical system. Projects that take 3 days to complete on the benchmark system will be awarded 330 credits.


BigWU Work Units get a 100% bonus on the credit, due to their size.


The operating system (OS) on which the client is run shouldn't be an issue anymore. A 2.8 GHz P4 running Windows XP will perform almost exactly the same as if it were running Linux, because the Cores share the same source code and libraries. This used to be different, when each OS used libraries and code specific to that OS.


Awarding Credit

When a client submits a WU, the credits are awarded to the username/team combination which was current at the time of completion. This means that the username and team configured in the client can be changed while a WU is being processed; it is not locked to the username and team configured when the WU started.


Once credit has been awarded, this cannot be changed. Therefore it is not possible to move previously earned credit from an old username/team combination to a new one. Keep this in mind if you decide to create a new username or join a different team.
See also: Altering Published Stats


Depending on the load of the servers at Stanford it can take some time after the completion of a WU for the credit to be added to the user who completed it. But most of the time this happens within an hour.


There are also cases where no credit is awarded. This is not communicated back to the client, who'll only receive notice that the WU was received. If you need to know why you were not awarded any credit you should ask on Folding-Community where a Moderator can lookup your submission with the Project, Run, Clone and Generation numbers of the respective WU.
See also: Special Folding-Community debugging output in qd, the queue.dat dumper.


In the unfortunate case that the WU is not completed but ended prematurely, for instance when an Early Unit End (EUE) occurs, credit is still awarded. A formula is used to determine the percentage of work which was completed before the EUE, and that percentage is awarded. Example: If you had completed 60% of the work and the WU was worth 100 points, you will be awarded with 60 credits.


Unfortunately there are some exceptions to this policy. When a WU has an EUE, some of the data will be incomplete. If the server doesn't get a valid report indicating that you have completed 60%, it will estimate how much it thinks you might have completed. These estimates are based on the amount of work a very slow PC would have completed. So, for those with faster hardware, the estimate will be significantly lower than you'd like.


Lost WUs

The term 'lost WU' is a coined term to represent WUs that have been finished by the donor FAH client, but have not been credited to the donor points statistics. Most WUs are actually not lost at all, but will receive proper credit during the next stats update. (This could take up to an hour or two, but is usually more often.)


Lost WU Recrediting

Ideally the finished WU will get back to the FAH servers and get the proper amount of points to "the donor account". If something goes wrong there are no automatic procedures to catch those errors, and anything "missing" must get reported to Folding Forum or to the project owner. If the claim can be backed up with some evidence (whatever it is - FAHlogs, global events, ...) then the project owner must MANUALLY recredit ANY missing WU.


There are no fixed rules on how or when or if recrediting must be carried out. The main focus is on the science and not on tinkering with statistics. As long as WUs are flowing back then all is fine. If the project owner has no spare time for recrediting then it will not happen at all or very late.


Note: The project owner can be found on the project summary page in the contact column.


Links

Personal tools