I've been working on a program for a while that downloads bulk files from Amazon S3 buckets.

If you don't know what they are here is a quick explanation.

S3 is the abbreviation for Simple Storage Solution and is a very cheap cloud storage service offered by Amazon under their AWS banner.

People use these for all sorts of storage, you can even run a static website from this service quite easily.

The thing about it is that there were over 2 trillion files stored there in 2013 and the numbers have only increased since then.

These files are stored in buckets which is the term Amazon uses for the segments that you create when you want to use the service.

These buckets can be set to be public or private and it turns out that somewhere around 7% of the buckets are public.

7% of 2 trillion is umm, lots and lots. 

The calculator says 140 million, that was in 2013, how many now?

If there has ever been a link to that file Google knows about it and you can find it with a good search.

Finding these files one at a time is possible but very tedious as is downloading them one at a time.

However, if the buckets are public putting the URL of the base account gives a list of everything in all the buckets in XML format.

This program takes that URL, filters out the file links and downloads them to the folder of your choice.

Sometimes there is a LOT of data, the last one I did gave me 2.2Gb of files.

Why would you want all this data?

That's a very good question and one my wife asked me as well.

The answer is that you don't necessarily want all the data, what you do want are the ideas behind it.

Of course, some of it is PLR, MRR or RR.

You can't use the RR stuff as is but you sure can with the PLR & MRR stuff.

Regardless of that you can use all of it as seed ideas for your products or curate it for blog posts etc.

So can you see how this will make life easier when you can get all this data?

Reply to this email if you're interested enough to have a look at it.

Regards,

Brent.

P.S.  While you ponder on the bigger picture of how you could use an almost unlimited supply of content and content ideas perhaps you'd like to have a look at how to use it.

As I said in yesterday's email there is a training module that will help you make good use of the PLR you already have and give you ideas on how to use the extra should you choose to go down that path.

You'll find out more here.

You're getting this email because you signed up on my
website or bought something from me.

Unsubscribe here