Limitations¶
DataManagement¶
LFN length¶
Because they are stored in a database, the LFNs are limited in size. The standard size is 255 characters. It is enforced in the following database:
- JobDB
- TransformationTB
- StorageManagementDB
- DataIntegrityDB
- FTSDB
- RequestDB
Shall you want to have longer LFN, then you would need to update your database manually.
A special case is the DFC. The limitations depend on the Directory and File managers you use.
In the DirectoryLevelTree and FileManager (defaults one) managers, the LFNs are split by ‘/’, yielding other limitations:
- 128 char for the filename
- 255 char for each directory level
In case of the Managers WithPkAndPs (LHCb):
- 128 char for the filename
- 255 for the base path
Scaling¶
Servers¶
When you servers are heavily loaded, you may want to tune some kernel parameters. Internet is full or resources to explain you what you should do, but a few parameters of interests certainly are the number of file descriptors allowed, as well as a few kernel tcp parameters that should be increased (https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt):
net.nf_conntrack_max
net.ipv4.tcp_max_syn_backlog
net.core.somaxconn
net.core.netdev_max_backlog
Finally, the parameter SocketBacklog for a service can be increased (man listen is your friend).
Duplications¶
In some cases, it is possible to run several instances of the same agent/service in order to scale.
Services¶
| System | Component | Duplicate | Remarque |
| Accounting | DataStore | PARTIAL | One master and helpers (http://dirac.readthedocs.io/en/latest/AdministratorGuide/Systems/Accounting/index.html#id3) |
| ReportGenerator | |||
| Configuration | Configuration | ||
| Server | PARTIAL | ||
| DataManagement | HttpStorageAccess | ||
| DataIntegrity | YES | ||
| FileCatalog | YES | ||
| FileCatalogProxy | |||
| FTSManager | YES | ||
| IRODSStorageElement | |||
| StorageElement | |||
| StorageElementProxy | |||
| Framework | BundleDelivery | ||
| ComponentMonitoring | |||
| Monitoring | |||
| Notification | |||
| Plotting | |||
| ProxyManager | YES | ||
| RabbitMQSync | |||
| SecurityLogging | YES | ||
| SystemAdministrator | YES | In principle there should be one on each and every machine | |
| SystemLogging | |||
| SystemLoggingReport | |||
| UserProfileManager | |||
| Monitoring | Monitoring | ||
| RequestManagement | ReqManager | YES | |
| ReqProxy | YES | ||
| ResourcesStatus | Publisher | ||
| ResourceManagement | |||
| ResourceStatus | |||
| StorageManager | StorageManager | ||
| Transformation | TransformationManager | ||
| WorkloadManagement | JobManager | ||
| JobMonitoring | |||
| JobStateSync | |||
| JobStateUpdate | |||
| Matcher | |||
| OptimizationMind | |||
| SandboxStore | |||
| WMSAdministrator |
Agents¶
| System | Component | Duplicate | Remarque |
| Accounting | NetworkAgent | ||
| Test_NetworkAgent | |||
| Configuration | Bdii2CSAgent | ||
| GOCDB2CSAgent | |||
| VOMS2CSAgent | |||
| DataManagement | CleanFTSDBAgent | NO | |
| FTSAgent | PARTIAL | See bellow | |
| Framework | CAUpdateAgent | ||
| MyProxyRenewalAgent | |||
| RequestManagement | CleanReqDBAgent | NO | |
| RequestExecutingAgent | YES | ||
| ResourceStatus | CacheFeederAgent | ||
| ElementInspectorAgent | |||
| EmailAgent | |||
| SiteInspectorAgent | |||
| SummarizeLogsAgent | |||
| Test_EmailActionAgent | |||
| TokenAgent | |||
| StorageManagement | RequestFinalizationAgent | NO | |
| RequestPreparationAgent | NO | ||
| StageMonitorAgent | NO | ||
| StageRequestAgent | NO | ||
| Transformation | InputDataAgent | ||
| MCExtensionAgent | |||
| RequestTaskAgent | |||
| TransformationAgent | |||
| TransformationCleaningAgent | |||
| ValidateOutputDataAgent | |||
| WorkflowTaskAgent | |||
| WorkloadManagement | DiracSiteAgent | ||
| JobAgent | |||
| JobCleaningAgent | |||
| PilotMonitorAgent | |||
| PilotStatusAgent | |||
| StalledJobAgent | |||
| StatesAccountingAgent | |||
| StatesMonitoringAgent |
FTSAgent¶
This agent can be split in two: one agent for the failover transfers, and one for the others (coming from transformations and so on). For this you need to define two agents using both the FTSAgent module, and use the ProcessJobRequests flag: once to True, once to False.