How to store billions of tasks?

Imagine that you’re living with one hell of a crazy wife. Every day she’s giving you bunch of tasks. “Mow the grass”, “water the plants”, “take out the garbage”, “replace the light in the kitchen”, “build a fence” etc. For every task you complete, she goes bananas and create 10,000 more on the spot, all related to the task itself (“build a fence” leads to “paint the fence”, “put a nice sign on the fence”, “practice your wax-on, wax-off” … you get the drift). Reluctant to perform all of these tasks but smart enough to know that you’ll lose at least 50% of your property (divorce are nasty), you start collecting these tasks (you write them on papers). You take one paper (single task), perform it and return to your wife with a big-fat smile of your face. She, in return, creates 10,000 new related tasks to the one you’ve just completed, write them on paper and put it in a BIG box. Every once in a while she’s not waiting for you and adding “main” tasks to the box by herself. After performing one task, you pick another one from the box (FIFO), without knowing if it’s a main task or not, you go to your way “eager” to perform that task as requested (as if you have a choice). This goes on and on and on…

The box is the tricky part here. Can one box hold billions of papers? hardly. So you start collecting boxes and now it’s getting harder as you need to add new boxes when needed, find the “right” box to pull tasks from and making sure these boxes won’t break (maintenance) with time.

Here are a few assumptions you can take as is:

1. Your wife is looking for perfection but only in the “main” tasks which means that if you were to build the fence, you have to do it perfectly so each mini task related to it is crucial.
   
2. Your wife tends to forget things so you can assume that occasionally, she’ll add new “main” tasks that were already performed or exist in a different box.
   
3. You can’t throw away tasks “just because” as you don’t know if a “mini task” will be thrown by mistake (= your wife will be pissed off. 50% is gone.).
   
4. Be cool – you won’t perform the same task twice. This one is on me.
   
5. Drinking RedBull (or XL or whatever energy drink you’re familiar with) 24-7-365 you don’t need to rest. You don’t need to sleep. Think robot (funny combination, for 2007).

How to store billions of tasks?

I kinda like the “green feeling” of a forest. Oh right, we also need them in order to breath (El Gor is more convincing than I. Thank God). Most importantly, it costs a lot of money buying so many papers! And the boxes!  You’ll need a lot of green ones ($) – not trees but we can’t “breath” without them as well). Oh well… You’re starting to build a “Boxing mechanism”, hire a few guys to maintain them, getting a VC to give you some extra $$$ and after a few months\years you got it cover!

What do you do if you don’t have the extra $$$ or more important the extra time to develop this kind of storage system?

How to store billions of tasks? You don’t. You can’t. 
In most scenarios, when things seem too difficult to accomplish (with the given limits) try a different angle: “If you don’t like the answer, ask a different question”.

We know that each task creates a lot of new related tasks right? We also know that keeping those new tasks is the tricky part (the BIG box problem) so what else can be done? Let’s change the question. “How do I make my wife happy?” seems like a smarter question. If it’s expensive to save those tasks why not doing these related tasks on the spot instead of storing them in the box(es)?  How is this going to help us? Now we can throw away tasks because these tasks will be added later on (assumption #2. Thank God your wife is not the robot). Assuming that we can save about 1,000,000 papers in one big box – we’re all set. If the box is full, we’ll simply throw away new “main” tasks, feeling good as we KNOW that we’ll get to them later on (again, assumption #2). Now all we need is one simple box with a limited amount of papers. Less $$$ to waste and much simpler storage system to develop.

Crap, it just hit me. I’m doing some cool sh$% at Semingo! Join us!