General Discussions

Knit Engine [Ver. 0.1]


Indian Bloggers

“It could just be a keyword. More ‘Just’ will be done to it, after giving it to knit.”

 It was a year passed my research program and I had stopped living my dreams. Mostly because I knew my research work was ONLY going to hit the library shelf under ‘thesis section’ and may be find few citations at the corner bends. I so much not wanted that. While I entered the program with Turning Nomination in mind, not sure if I was capable of that, but giving a try was no harm anywhere.  I did not feel any guilt in scraping out my one year research work (which according to my guide I had not done anything and he was right; may be) to switch to something that really took away the sleep and gave the feel of actual research.

Knit Algorithm, yes. It made me forget the snooze button. It gave me the insight into real research. Not bothered of where it would lead, it just felt curious enough to give a start. Here I am, after one and a half years, presenting what ‘we’ have achieved so far.

Introduction
We have been using search engines and it has also seen an intelligent make-over in the name of semantic engine. Knit is not a search engine of any kind. It cannot answer ones queries. All it can do is for the given a keyword, present the related data to it. It might surprise one for the amount of varying related data it can collate. The idea of knit is not to search the data on web instead to knit them.

Do you see the relation between a teacher and a pen? A TV and a TV stand? Cake and cream? Bike and shirt? Pen and Bean bag? Trimmer and Shampoo?  Well, knit sees it too and many other surprising relations, not just that. Yes, it’s a probability distribution of related data where the upper threshold is presented to the user. So, basically knit is a self learning recommendation engine.

Why Knit?
Web has a lot of data. It has a lot of related data and most uses are still not aware of it. A user might not even know for the keyword he has, the amount of related data present on the web. What if user gets the related data knit together in the form of layered umbrella? That is what knit-engine does.

What does Knit do?
It ties and presents the related data together. It crawls, scrapes, searches, gathers, relates, recommends, clusters, mines, associates, retrieves, learns, then a lot of math and *Eureka*.

knit-engine
High Level Knit Architecture

The high level architecture is presented in the figure above. There is a crawler to gather data, then some processing happens on it and hands it over to knit engine. Based on the user input data, after processing it, an environment is prepared and compatibility iterations are run. Results are presented to the user and the algorithm learns through every further click on the result.

What does Knit Engine Ver. 0.1 do?
The current version does the following:

  • The ad-hoc crawler has pulled 28,000 data sets, pre-processed as per current needs and loaded to knit-table
  • Knit Algorithm accepts input and prepares the environment as per input needs
  • Does some math over six compatibility iterations and gets the related items
  • Brings the smiles and happiness

Life-Long Learning
While a lot of machine learning algorithms use the existing mining algorithms and add a feedback control, Knit algorithm is more of a life-long learning algorithm. It has a design where code learns and updates itself based on the current trends. Knit currently does not use any kind of utility matrix or user preferences to gather the related data.

The first version release has fewer iterations running to achieve the required task. There is lot in the design yet which has to be realized through implementation. However this first release was essential to give a go and a motivation for the further research.

The Support

research-support

This research is currently supported by theory from 37 research papers, 18 coursera courses, my old machine, reworked designs, scribbles, Python, and a lot of self motivation. I would take all the opportunity to mention and thank Vishwanath Telsang for being there and supporting all through the journey by far. When I mentioned ‘we’, it was about me and him talking. Would also like to thank Anirudh Bengeri for seeding this idea into my brain and for now it has turned into a reliable sapling.

What Next?
Eager to see the day when knit-engine will go live. With two people working, it might take time but the hopes are definitely high. I hope this work will further give a lot of research interest in the academics and as well as in industry. For now looking for all kind of support to keep it ON and on the GO.

Release Notes
April 24, 2017,
The release of Knit-Engine Ver. 0.1

Package details:

  • Code: 2400+ lines
  • Language: Python
  • Minimal document support with necessary designs
  • Relevant Courseara course Notes
  • Summary of 37 research papers

Note: The implementation is currently available for personal in-person demo only for numerous reasons.

Contact Email: prakash.hegade@gmail.com

Advertisements

9 thoughts on “Knit Engine [Ver. 0.1]

  1. Happy to see you progress. The idea is great and can have many applications in the real world.
    One question.. Can it handle the data in real time and what is the computational delay?

    Liked by 1 person

    1. Thank you so much. I am happy to hear that. 🙂

      Yes. It can handle data in real time. For now there are only 6 iterations implemented and the data set is around 30k. The delay is in the unit of few seconds. Have not implemented any of the efficiency terms yet.

      🙂

      Liked by 1 person

  2. Sir I think it’s a brilliant idea! The concept is amazing.The idea itself shows how complex it must have been for you to think about it and come to a blueprint level in the first place and the rest is sheer magic. It’s incredible to see the flow already. I have understood the concept to a considerable level, but I am not able to connect in what all ways this can be useful. It is a brain teaser!
    Wish you all the very best sir, waiting for it to go live! Kudos to the decision of accepting risk and working your way 🙂

    Like

    1. It did take a lot of time to freeze certain designs and still many are yet to be done. Yes, I have been very happy as well with the progress, though taxing at times.

      Applications, Making an ebook of it this month. There are lots of them. 🙂

      Thank you so much.. 🙂 Happy to hear that. 🙂

      Like

Let me Know What you Think!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s