I been all day in nonresting times studying to understand transformer code usage and also pytorch and also numpy. 

but there is something weirdo: 

e.g. bert input considers paragraph or sentence scope be of 512 length (tokens) 

and there is feature interim result that encoder generates. 

but then the dropout functionalities are written considering the actual paragraph/sentence lenght which is not 512. I initially modified to have them be extended stupidly but thats illogical for at least the feature vector.

I had searched for a library which does constituency parsing and dependency graph generation and currently working to understand that. to understand transformers more and learn its libraries (pytorch and transformers) but there is such inconsistency. 

I would eventually construct a neural network model after i learn from such examples of pytorch/transformers that does constituency parsing.

but getting used to this advanced indexing schemes also takes time. e.g. i mean usual tensor slicing/indexing methods. 

I think I lack quite expertise in writing neural networks. cause not used to tensors methods yet but in some time I would get used to I guess but definitely not any proficient in. 

e.g. its still very hard to create specific copy operations from one tensor to another. 

I remmeber doing such tensor copy operations that in pandas or numpy before but i dotn remember exactly how. 

so its like i think to be proficient in this indexing/copying/transposing etc methods would take some slght time.

Currently the neural network is running, but i dont think it would work correct. cause feature drop out is completely incorrect now due to fast refactoring to have matrix multiplications error disappear. but actually currently drop out is working only for existing tokens part. I dont understand why they had written a code where encoded features are taken of original paragraph size but not pretrained BERT's input size which is 512. code is incorrect i mean it actually does not work with Bert. but  I did such fix which is incorrect also to fix such matrix multiplication error but still very illogical. 

I think i would learn transformers more and these libraries. Thereby I would write my transformers based code but of course it would take some proficiency building period maybe a day or two. So I currently would check accuracy of this neural network solution but I would rather build my solution 1 or 2 days later. (this code is unnice.)

so following project challenge would be:   getting more used to tensor syntax/semantics/ neural network library/transfoermers then writing the graph dependency extractor myself.  

so from today's project studies: 

so today I am happy that I slightly got introduced myself to tensors/numpy and transformer methods.

but the q k v part of transformers i had not very much in detail analyzed. of attention mechanisms. I mean from example projects but I would analyze in detail. 

but I couldnt any start studying topology grammar task. so today were not any efficient ai project day. but still not fully unsuccesful day, at least I partially understood transformer usage methodology and partially revised numpy/tensors knowledge.  

I thought I would finished this running neural network task (by fixing its errors) by noon and its 1am now. I rested alot either. but i mean its not the original project plan for today. it were anticipated to be more spared to topology studies for today instead. but I got to understand that I need to take /learn proficiency in these tensors/numpy topics again.  but not tomorrow.

I wish to not postpone alot topology study and would spare topology grammar study to entire tomorrow. 

hmm but then maybe at times, I would do exercises with numpy and pytorch/tensorflow to get more proficient in tensors topics. I definitely lack proficiency in this to tackle the following later tasks so thereby I anticipate to spare some time also on tomorrow (but tomorrow not much) and monday such exercises with numpy and tensor libraries to get proficiency to be easily code/maintain the neural networks necessary in project.

so then tomorrow would be mainly topology grammar studies. and if possible to start with PTB PoS tags conversion to such topological RDF language.  of course language would be created iteratively either. initially simple topology language.

so tomorrow wishfully i could at least have a very draft version of iteration 1 of the topology RDF language  and start representing dependency graph constructs tags and so. from either Brown corpus like sets also from PTB like corpus. 

yayyy so by monday, wishfully I had iterated alot in ai project: at least i ran neural network based on Bert but it has issues and such nn module would be rewritten entirely later in following week's evenings ai project study time. 

hmm at the same time,  in the holiday i finished the computational linguistics course studying last day. and learnt about TBs.

( ayy again music player i forgot a soundtrack whilst headphones were off and it started randomly playing music from internet :S  and figure out it played a sound track that i didnot liked :S unnice :S i dislike autoplay feature. I need to check itnernet to learn how to close autoplay. )

continuing ai proejct topic:

continuing the project accomplishments:

I think if I could craft a draft topology grammar and RDF tomorrow working hardworkingly and if i could also convert at least some of PTB tags translations to such RDF, I would think it would be nice. 

then in weekday evenings I could continue hmm hardwork of getting proficient in tensor language/numpy. to write the dependency graph /constituency info module by reading/checking existing algorithms to craft/try new algorithms. (based on transformers alike Bert etc)

so by then if i reach next week's friday with becoming proficient in tensor libraries/numpy and transformers library, and also have a basical topology rdf that translation of PTB tags is in progress, it would be nice accomplishment if i could be in such state in next week's friday.

hmm then would work more to topology RDF translations/topology language on next weekend also. 

hmm then other week  I guess i would start working on ontology signatures binding tasks. with transformers even etc. then also paragraph sentence continueation's constituency information etc like tasks. so by 2 weeks later i think iw ould be working to such tasks. by 3 weeks later I wish such tasks are finalized. hmm yepp.  so roughly i think by June 1 or June 10 or so, the first version then would be ready. then topology language updates/query languages to toplogy layers etc etc would be studied to with along wiht some additional phd studies.