Pondering About Language and Shopping

AI in french becomes IA. This is not to say that French and English mirror one another, but Artificial Intelligence is equivalent to “Intelligence Artificielle”. Not all terms used in Neural Networks are as easily equatable in French, however. They certainly are not literal translations, and have their own separate set of jargon. For instance, the word “pondération” may appear as though it might be related to thinking or thoughts, but in the context of Neural Networks, it refers to what are known as “weights”, which do not truly have much to do with quiet ponderings. Where does this come from? The answer to that question lies in a Belgian website, known as “Elements of AI”. The title is the only part of the entire website written in a language that is not French. They are essentially a Belgian team trying to make concepts of AI more accessible to The People, explaining them in an easy-to-understand fashion. After coming across this website, filled with amazing illustrations and truly beautiful examples, with carrots and potatoes and shopping, I could not resist reading the articles. This is where I came across “pondérations” and “combinaisons linéaires”, the latter being linear combinations.

I had an “Aha!” moment while reading about linear combinations in terms of shopping. I will not describe the actual analogy, but what I have been trying to remember and picture in my mind ever since I watched the Linear Algebra lecture series by Dr. Gilbert Strang, was simplified after a good deal of lingual complications. The actual content of the article did accomplish its goal - to make AI more understandable, save for the lingual barriers I had to cross with my lack of fluency fragmenting the phrases. Regardless, I understood, finally, how and what exactly “weights” are and how and why they are multiplied to the input vectors and what it really means to adjust a weight. What was a vague understanding became much more clearer, and I believe I am now closer to really simplifying machine learning to math. What seemed thickety yesterday is now a series of clear trees, albeit a little tangled.

With newfound confidence, I decided to leap into the next Phase of my Original Work, which included adding positional encodings to my word embeddings. The idea was to provide more information about how closely-related words by meaning are located with respect to one another. So, distance and meaning relatedness. However, I did not understand why sine and cosine functions were used as part of calculating the distances. Therefore, this is my question of the week, and I will try to better understand what role these trigonometric functions play in determining relative positions of words within a sentence.

While creating Positional Encodings, I found the opportunity to explore Python from its perspective, and not simply a means to a solution. One of my coding best friends has always been StackOverFlow, ready with answers to my questions asked by at least one other fellow coder. My relationship with StackOverFlow was always simple, analogous to the way our fingers ache for the nearness of Google. I would be working on a problem, and while programming the solution, I would realize that I did not know how to achieve a certain action. Then, I would search if this microscopic action can be accomplished in the said language in an efficient manner. This was not searching for ways to solve an actual problem, or even looking for algorithms that help, but the actual syntax involved. In this way, I learned a majority of the little programming I know.

However, there is an issue with this method of learning a language - really any language. This is similar to searching word-to-word translations to convert a phrase in one language to another using a dictionary - a solution that is neither accurate, nor effective, nor efficient, nor lingual. Besides, in normal languages, we have idioms that are special because of linguistic cultures, and we might translate a phrase into something that unintentionally conveys the wrong message. I noticed this can be applied to programming languages as well. By continually searching for only what I need, and using the learn-on-the-go method every time, I realized I was severely limiting myself in terms of learning about all the options available. Having a list ready before going shopping might be efficient, but it also reduces the opportunity to explore all there is to see. In this case, although it is important to emerge successfully out of a programming adventure, the real goal is to learn.

In practical terms, there are certain Pythonic idioms that I came across, idioms that are fundamental and quite basic, but important and interesting nevertheless. For instance, to swap the values of two variables, there is a naïve approach, which would take three variables and five lines of code. Or...there is the Pythonic way to do it, which looks much more elegant, beautiful, and lovely. The Pythonic way takes no extra variables, and can be done in three lines of code. It also is more legible, and as I said, more Pythonic. This is one of the many reasons why Python is much better than Java (my personal opinion). Regarding my Original Work, I had to apply a sine function to only the even columns, and cosine for the odd columns. I could do this the boring way, which would entail checking whether the column index was even or odd, and traversing through the matrix. However, I learned a really elegant trick (the Pythonic way). Python lets me select certain sections of any matrix with double colons. I can simply indicate a starting point and how big the skip should be (how many units to skip over by), and “voila”! This made me realize the importance of lingual idioms, even if the language is a programmatic one.