hey last day's records entries to tblae has been like 23.104.000.000.000 resultant records in 7:30 hours (ina  50 node cluster with doing parsing with Jena parser of ttl) .  hmm but there are duplicates alot. due to tumbling window approach. would reduce the row count later.

wov 23 billion records results of jena parser run. nice. :) I mean even its not very neat the code notebook to run this parsing ontology db is working nice imho even at 7:30 hour somehow its says cluster terminated.

i am happy of managing to setup zeppelin notebook codes to handle this huge data silo tasks like this. 

hmm last days has been like 200 euro extra bill due to this task but its ok its 23 billion row data that is parsed with jena. 

I really like aws very much :)  I mean being able to spawn 50 cluster nodes at once :)  and procesisng data silos at once :) nice :) 

 

I mean its kind of affordable cluster solutions. i saw other clouds big data spark solutions and they were very expensive.  but take this for instance spawning a 50 node cluster for 7:30 hours or slightly more due to debugging or so, it just only takes 200 euros. its very cheap with respect to other cloud solutions I saw. plus you can instantly easily spawn a cluster from ui. 

I once worked with aws cloud before in a startup and liked there. and seeing another cloud solutions I rethink i think aws is least expensive cloud solutions for big data/ml.

its just totally amazing to be able to spawn an 50 node cluster just in some minutes.

 if it were one other cloud, it would take 10K doillars to spawn this much cluster.  I seen other clouds spark big data solutions and i think theirs 50 nodes would take at least 10K euros imho.

aws is like having alot  ml/big data environment with very low expense. one of the best things that technology priovided to we ml big data engineers hobby projects is this not very expensive big data environments. 

since we cant' setup buy 50 nodes of pc to home to do hobby projects to do ontology processing/knowledge engineering. plus its very expensive in other clouds to setup spawn 50 nodes cluster or so.  

5 years ago I were working in a  startup with aws cloud solutions by then, and have chance to work again in this hobby project, and i definitely think its awesome cloud solution again.  its like being able to spawn 50 nodes, 100 nodes clusters in minutes is just so awesome imho.

---------------------------------------------------------------------------------------------------------------

hmm checked other cloud operators expense calculators, and it seems as there is not like 50 fold increase but something like 3 or 4 times expensiveness. 

not checked all cloud solutions pricing either. but seems aws is really very cheaper than at least one cloud solutions I checked. 

it just is very amazement to see as a data engineer an 50 node cluster gets spawned in minutes and do process big data silos to do build ai projects. its just, aws is really amazing to people whom dont have alot  budgets to build ai projects. nor i think for startups either that's cool. its just like a bliss aws existince in cloud solutions domain to people whom budget means importance a lot.  other cloud system's web ui's are even not any nicely css'ed as aws either imho. its like empowerment of people/startups to do big data projects at home also that people whom might lack such on premise internal cloud solutions capabilities. its amazing i think. 

i as a hobbiest ml engineer I think aws is like a bliss :D it just empowers buiidng ai projects at home since its at least 30%  cheaper as i checked now . i mean to people like us that dont hve alot budget to ai projects,  200 euros versus 600 or 700 euros means alot. it means empowerment to build ai projects. to any ai hobbiest there. its amazing. 

------------------------------------