“It could just be a keyword. More ‘Just’ will be done to it, after giving it to knit.”
It was a year passed my research program and I had stopped living my dreams. Mostly because I knew my research work was ONLY going to hit the library shelf under ‘thesis section’ and may be find few citations at the corner bends. I so much not wanted that. While I entered the program with Turning Nomination in mind, not sure if I was capable of that, but giving a try was no harm anywhere. I did not feel any guilt in scraping out my one year research work (which according to my guide I had not done anything and he was right; may be) to switch to something that really took away the sleep and gave the feel of actual research.
Knit Algorithm, yes. It made me forget the snooze button. It gave me the insight into real research. Not bothered of where it would lead, it just felt curious enough to give a start. Here I am, after one and a half years, presenting what ‘we’ have achieved so far.
We have been using search engines and it has also seen an intelligent make-over in the name of semantic engine. Knit is not a search engine of any kind. It cannot answer ones queries. All it can do is for the given a keyword, present the related data to it. It might surprise one for the amount of varying related data it can collate. The idea of knit is not to search the data on web instead to knit them.
Do you see the relation between a teacher and a pen? A TV and a TV stand? Cake and cream? Bike and shirt? Pen and Bean bag? Trimmer and Shampoo? Well, knit sees it too and many other surprising relations, not just that. Yes, it’s a probability distribution of related data where the upper threshold is presented to the user. So, basically knit is a self learning recommendation engine.
Web has a lot of data. It has a lot of related data and most uses are still not aware of it. A user might not even know for the keyword he has, the amount of related data present on the web. What if user gets the related data knit together in the form of layered umbrella? That is what knit-engine does.
What does Knit do?
It ties and presents the related data together. It crawls, scrapes, searches, gathers, relates, recommends, clusters, mines, associates, retrieves, learns, then a lot of math and *Eureka*.
The high level architecture is presented in the figure above. There is a crawler to gather data, then some processing happens on it and hands it over to knit engine. Based on the user input data, after processing it, an environment is prepared and compatibility iterations are run. Results are presented to the user and the algorithm learns through every further click on the result.
What does Knit Engine Ver. 0.1 do?
The current version does the following:
- The ad-hoc crawler has pulled 28,000 data sets, pre-processed as per current needs and loaded to knit-table
- Knit Algorithm accepts input and prepares the environment as per input needs
- Does some math over six compatibility iterations and gets the related items
- Brings the smiles and happiness
While a lot of machine learning algorithms use the existing mining algorithms and add a feedback control, Knit algorithm is more of a life-long learning algorithm. It has a design where code learns and updates itself based on the current trends. Knit currently does not use any kind of utility matrix or user preferences to gather the related data.
The first version release has fewer iterations running to achieve the required task. There is lot in the design yet which has to be realized through implementation. However this first release was essential to give a go and a motivation for the further research.
This research is currently supported by theory from 37 research papers, 18 coursera courses, my old machine, reworked designs, scribbles, Python, and a lot of self motivation. I would take all the opportunity to mention and thank Vishwanath Telsang for being there and supporting all through the journey by far. When I mentioned ‘we’, it was about me and him talking. Would also like to thank Anirudh Bengeri for seeding this idea into my brain and for now it has turned into a reliable sapling.
Eager to see the day when knit-engine will go live. With two people working, it might take time but the hopes are definitely high. I hope this work will further give a lot of research interest in the academics and as well as in industry. For now looking for all kind of support to keep it ON and on the GO.
April 24, 2017,
The release of Knit-Engine Ver. 0.1
- Code: 2400+ lines
- Language: Python
- Minimal document support with necessary designs
- Relevant Courseara course Notes
- Summary of 37 research papers
Note: The implementation is currently available for personal in-person demo only for numerous reasons.
Contact Email: firstname.lastname@example.org