Estimating memory requirements for job submissions

[Psirving]

I'm wondering if there is a way of estimating the memory requirements for an instance of DanceMapper based on read depth and target length. I've been requesting 10G, but this usually isn't enough for when I have >1M reads. I've also noticed that memory tends to run out during the RInG and PAIR analysis, and not the DANCE analysis, which is not what I would've expected.

Another student I was helping ran out of memory with a 30G request. He had unusually high read-depth.

Any advice?

[anthonymustoe]

Yes, memory is a major issue, and RING and PAIR analysis are the most memory intensive. Your suggestion about building estimate tool is a good one and something I will add to the list.

The simplest fix for the time being would be to undersample reads, but this will reduce power for RING/PAIR.

I don't have a firm estimate of memory demands right now, but it scales as N x R, where N is RNA length and R is # of reads. If I had to provide a heuristic, I would say Mem = 50 x N x R bytes? So for 400 nt long RNA with 1M reads, that comes out to 20 GB. But that is just my gut feeling right now.