Calculating Disk Requirements
The basic rule of thumb is that ELSA will require about 50% more disk than flat log files. This will provide archived and indexed logs. Archive logs require about 10% of flat file logs, log indexes require 40-50% more disk than the flat files, so together, there is a roughly 50% overall penalty.
To specify how much disk to use, see the config file entry for log_size_limit, which is the total limit ELSA will use. Within that limit, the archive section’s config value for “percentage” dictates what percentage of the overall log_size_limit will be used for archive, and the rest will be used for indexed logs. If you do not wish to archive, set the percentage to zero and all space will be for the index, or vice versa.
Choosing the right Hardware
Single ELSA node will comfortably handle about 10,000 events/second, sustained, even with slow disk. As shown above, ELSA will happily handle 50,000 events/second for long periods of time, but eventually index consolidation will be necessary, and that’s where the 10,000-30,000 events/second rate comes in. A virtual machine probably won’t handle more than 10,000 events/second unless it has fairly fast disk (15,000 RPM drives, for instance) and the disk is set to “high” in the hypervisor, but a standalone server will be able to run at around 30,000 events/second on moderate server hardware.
Recommendation is minimum of two cores, but as described above, there is work enough for four. RAM requirements are a bit less obvious. The more RAM you have, the more disk cache you get, which helps performance if an entire index fits on disk. A typical consolidated (“permanent”) index is about 7 gigabytes on disk (for 10 million events), so I recommend 8 GB of RAM for best performance, though 2-4 GB will work fine.
<br< RAM also comes into play in temporary index count. When ELSA finds that the amount of free RAM has become too small or the amount of RAM ELSA uses has surpassed a configured limit (80 percent and 40 percent, by default, respectively), it will consolidate indexes before hitting its size limit (10 million events, by default). So, more RAM will allow ELSA to have more temporary indexes and be more efficient about consolidating them.
In conclusion, if you are shopping for hardware for ELSA, you don’t need more than four CPU’s, but you should try to get as much disk and RAM as possible.