Posts

Showing posts from 2019

December Already

Hello everyone. It is December already and 2019 is almost over. Wow. It is the last day of Thanksgiving Break, and I had a nice day today. We are in the midst of rainy weather and cold temperatures, and it doesn't look like it'll be clearing up for the rest of the week. Today was a good day for me because I got work done and had quite a bit of fun. Now I just ate good food and am relaxing before I go to sleep. I don't have much more to say. Hope that I can continue maintaining productivity and satisfaction with myself as I did in the past. Still not perfect, but I will continue to work on improving my reflection and following through with my goals. Because life is good right now and I don't have much to say, I'll end the post here.

Season's Greetings

Hi all, Nothing formal. I just wanted to update this blog since it's been a while. Amazingly, just another month of college applications remains. I am now sitting on the computer, writing a blog post, for which I have very little idea. One thing I want to improve about myself is my lifestyle. I have a problem of getting easily distracted by the Internet and computers. I want to get to a point where I have established for myself a means to stay busy. Whatever it be, I just don't want to spend my time aimlessly watching videos or browsing online. I seek to do what I do with a purpose. At this point, I am somewhat happy with my lifestyle. I write in my journal daily, which gives me a record of the things in my life. For the past 6-7 weeks, I've been running 3 miles a day, which gives me a reason to be happy and maintain discipline (although I have missed more than the past week due to sickness and recovery). I have made a conscious effort to withdraw from my addiction to...

Changing Colors

Hi there! It's been a while since the last post, so I might as well discuss what I've been up to lately. School started roughly a month ago, and I'm enjoying the classes so far. I have also begun applications for college, which is simultaneously exciting and frightening. For Underground School, I posted some new videos based on the topics we are covering in math and physics. In physics, we are currently discussing the concept of relativity, which is interesting because we have the opportunity to put ourselves in the shoes of physicists about a century ago, and to vicariously experience the ingenuity behind some of the thought experiments conducted by physicists like Lorentz and Einstein. Other than that, things are going pretty okay. I ran a mile and a half with my friends earlier this evening around the neighborhood interior circle. I also climbed the biggest tree in the park without height assistance, which is nice. Today, I had little homework, so I worked on editing som...

Hello August

Hello everyone! I realize that I haven't committed to my promise of one-post a day, and this is actually the second time I've broken my promise. I guess I got bored of writing. I ran out of ideas, and didn't have much motivation to continue writing. However, I also recognize the long-term benefit of writing and preserving these memories, hence why I'm writing right now. As summer vacation comes to a close, I must say that I enjoyed this one. I had the chance to work, and had twice as much fun. I'm grateful to have had an internship this year. My last summer vacation was wracked with guilt of not working, as I had been rejected from all the places I applied to. I wanted to work, but wasn't driven enough to follow up on my goals. I think if I had been given a similar opportunity to work and learn from others, I'd have enjoyed it. Thankfully, I have more to show this summer, and was able to make this summer more productive and guilt-free than the last. - Hipp...

Underground School

I haven't officially introduced my YouTube channel on this blog, so I figured I might as well do it here. I started the channel in the beginning of June with the goal of spreading knowledge through Hindi lectures/videos, mainly regarding academic topics, such as math, physics, and computer engineering. By making videos, I hope to myself learn more about such topics, and to grow the channel. Thus far, the channel is performing reasonably. At the moment, I have 17 subscribers. Most of my series videos don't get many hits, so I should probably revamp my teaching style or the pertinence of the content I post.

Shoutout from Black Pen Red Pen?!?!

What's up guys? Something crazy happened today. So I went on YouTube and randomly stumbled upon BPRP's live stream. During the stream, there were about 40-70 people, and not that crowded, and it was an informal discussion with Lars, one of BPRP's viewers battling cancer. I started asking questions to bprp and Lars, and asked a few questions. I've been watching the channel's videos for a while now, and I hardly thought that Steve would read aloud say the name "Underground School". Given that this guy has over 300K subscribers, this opportunity was truly a blessing. Link: https://www.youtube.com/watch?v=kCmLsxq4d9w Check out the timestamps where he responded to my comments: 43:19 50:50 1:21:14 (shoutout!) 1:53:55

10 Things on my Bucketlist

I'm not really into posts like this, but eh...it's what Google recommends Idle for a day Lie submerged in water with an oxygen mask (maybe fall asleep) Undertaking a marathon Visiting Skydiving Undertaking a triathlon Meeting Sal Khan Moving to India Eating the world's hottest pepper Riding a jetpack

Another update...

Hello there, It has been two weeks since I published my last post... I'm not quite sure why I stopped writing, but I'll try to make it a habit to continue writing. Since I've missed two weeks, I plan to post twice daily for the next two weeks to make up for missed days. To be honest, I haven't really much missed writing. But commitment is commitment, so I will continue. Part of the issue is that I don't have many topics in mind to write about daily. My day today was pretty good, nothing much out of the ordinary. It's hard to believe how quickly this summer vacation flew by! I'm grateful to have had the opportunity to both find enjoyment in my work and play a ton. I found today that one of my neighborhood friends is moving out in a few months. I have few friends like him, so it saddens me a little to see him go. *sniffle* I had hoped that we could spend more time together and watch each other grow over the next few years, but anyhow. It was fun while it laste...

PyTorch vs Tensorflow

Today, I learned a bit about the difference between PyTorch and Tensorflow, so why not share? PyTorch and Tensorflow are two deep learning frameworks used in the creation of neural networks; they have largely the same function and intended purpose. However, the major difference between the two is the nature of compiling and computing computation graph. TensorFlow and Static Graph Construction In Tensorflow, a symbolic graph is created first, meaning that it represents all the operations that are to be conducted. Then, following compilation, actual values can be substituted, and numerical results for output can be computed. This is advantageous because the graph only needs to be computed once, rather than dynamically every time an output is to be produced. This is known as static construction, as the graph is created once and only once. PyTorch and Imperative Programming In PyTorch, however, the graph is created dynamically, meaning that there is no need to compile the graph upon...

Google AI Depth Prediction with Single Camera

I also recently learned that Google AI recently developed a neural network to generate predictions regarding depth:  http://ai.googleblog.com/2019/05/moving-camera-moving-people-deep.html . Interestingly, they trained the neural network on the hundreds of videos posted on YouTube pertaining to the Mannequin Challenge. The reason was that in the mannequin challenge, objects are stationary, enabling depth measurement using deterministic algorithms. They then train the neural network to match these values for a single frame. Another slightly more complex version of the model involved passing nearby frames before and after to the neural network. I think that's freakin' cool.

NASA iTech Forum

Yesterday, I had the chance to attend the NASA iTech Forum, where finalist entrepreneurs pitched their companies to NASA for evaluation. This event was part of a program that reached out to local companies, inviting them to present their products, which would potentially benefit NASA. There were a couple noteworthy presentations. One was by GoRout, a company that makes on-field wearable devices, specifically designed for football. The advantage to the solution was that it was portable, and can be applicable in a variety of environments. Another presentation was by Cluster Imaging, which demonstrated technology to generate depth measurements using two cameras placed a few centimeters apart, similar to the way stereo cameras work. Not that breakthrough. Baker Street Scientific also presented how they have 6000 algorithms that are good for processing data. Not any specific kind of data, just data. I didn't find that one very interesting. Ciao for now!

Breaking the Record!!

Image
I have great news! To the two, maybe three readers that actually see this post, you are witnessing history. This is my twenty-fifth post this year, meaning that I am matching my record thus far for most posts in a year. So far, it's been an engaging experience keeping this blog, as I'm able to write on topics I enjoy writing about, rather than sticking to strictly academic or formal writing, as is present in the majority of blogs prior to this summer. I struggle to find interesting and original topics to write about, but I hope to continue writing, as I see this as an effective way to not only reflect upon myself, but also develop my skills in writing about myself, which I anticipate will be a highly relevant skill as I enter my senior year of high school. Cheers!

Update

I thought it would be a good idea to take a few minutes to reflect, so here we go... It is currently 9 PM PDT on July 9. SAT Subject Test Scores are coming tomorrow, and I'll have the chance to see my scores on the chemistry and physics exams I took, which take a surprisingly long time to be returned. I wonder why that is. Anyway, today, I studied a little for the SAT, which I'm taking in August, and I completed Reading, Writing/Language, and Math (no calculator) sections. I plan to do Math (calculator) after completing this blog. Today, I'm grateful for the opportunity to have played outside with my neighborhood friends, for about an hour to 1.5 hour. We played soccer and chased each other around (okay, we were playing tag). I'm grateful for the opportunity to have enjoyed nature and feel good about myself. I'm grateful for my library, which is about a block from my home; I went to visit today, and although I didn't get any books, I'm glad I spent time t...

Tests

In the academic setting, tests are a means of evaluating a student's understanding of a particular topic. Tests are usually written, and carried out under time-restricted and stringent environments, and have become something that students dread. In this post, I'd like to propose a different (though not necessarily more practical) method of testing students and evaluating their performance. Here's the idea: the tests are carried out orally and in-person, one-on-one student to teacher. The teacher offers a problem to the student and asks the student to walk the teacher through how one would go about solving the problem. For example, if the problem is 2+2=?, the teacher would first present the problem to the student and ask him/her to solve it. The student would then proceed to explain from start to finish, how to solve the problem. The student might start by identifying the type of problem, and choosing a particular method to solve it. If the student falters, or is unable to...

The Nature of Consciousness

I am by no means qualified to speak on such a subject, but consciousness is a concept altogether unique. Consciousness is super weird in that it's hard to define. I may know that I have consciousness. I may know that my friend has consciousness. I may know that my dog has consciousness. But what does it mean to have consciousness? At its root, I suppose that consciousness comes down to being able to feel and to recognize how others' feel. Perhaps consciousness is an illusion. Life may as well be an intricately designed illusion, something like the world shown in The Matrix . But nonetheless, it is an illusion I believe in, that I have vested my thoughts, emotion, and energy into believing. I only know how I feel, but by observing myself and my actions, as well as others and their actions, I infer that those individuals possess similar emotions to me, based on the way they respond. Now we have come to the question, can a robot have consciousness? Now, reader, I have faith that...

The Kind of Life I Want

I want a life that gives me happiness. I want to live in a cycle of high days and low days, exciting days and dull days, days where I'm proud of myself and days where I'm dissatisfied. I want a life where I can exercise on a regular basis. I want a life free of addictions to time-wasters. I want a life where I have something worthwhile that occupies my time. I want a wife that I can talk to and cherish as my best friend. I want to have kids that open up to me a new perspective on life. I want to keep in touch and joke around with old friends. I want to continue to self-reflect and meditate. I want challenges but also hope that I can overcome them. I want difficulty and hardship in my life. I also want happiness and satisfaction in my life. I want a life where I can see the big picture and act on my decisions.

Neuroplasticity and Blindness

Neuroplasticity refers to the tendency of the brain to continuously change and adapt throughout one's life. While there is a higher degree of plasticity during childhood while the brain is still developing, the brain continues to adapt even through adulthood. Cortical remapping refers to a large scale remapping of the brain that occurs in response to a loss of particular sensation. For example, following an amputation of the right leg, the region of the brain in the sensory cortex that maps to the right leg is no longer relevant, leading to cortical remapping, or the reorganization of neurons in that region of the brain. Sometimes, cortical remapping can go wrong, leading to symptoms like phantom limb pain (PLP). Neuroplasticity occurs to a greater extent during young childhood, which accounts for the fact that children who go blind before 3 years or are born blind compensate with other senses. The brain is a magical place and adapts to our lives as we go.

John

John is a boy just like me, living in a parallel universe. He has the same life as me: same clothes, same school, same friends, same opportunities, same everything. Except there is one key difference: John is a different person. John makes his own decisions. When faced with the same problem, John and I may choose differently. This divergence is the source of our rivalry, ever since I was a toddler. Each of us has a counter keeping track of the total number of points we have accumulated. Any time I do something good, I would get a point. However, whenever John does something good, he also gets a point. Oh goodness! Unfortunately, I have been denied access to the counter, and I must only ascertain my relative standing through my parents. Some say that John is the perfect boy: at any given time, he does what he is supposed to do (although I say that independence and making mistakes leads to a richer life). I'm always in competition with John, and I'm sure he is as well; we'v...

How have you grown as an individual?

As I mentioned earlier, I don't particularly enjoy writing about myself, but this is a question I'll invariably have to answer at one point or another. So I might as well take a stab at it.

Schedule

It's summer, one of my favorite times of the year! It's a relatively relaxing and stress-free time of the year, and I have the chance to work on things I enjoy doing. Here's a rough breakdown of my schedule each day. 7:45 AM - Wake up, Brush, Breakfast, etc. 8:00 AM - Begin work on internship/other activities (usually with several breaks in between) 12:00 PM - Lunch 6:00 PM - Walk my dog and play outside 8:00 PM - Go to the gym (past few weeks I've been doing 5K, 100 situps, 100 squats, 100 pushups) 9:00 PM - Shower 9:30 PM - Dinner 11:00 PM - Go to sleep As I mentioned, this is a very rough breakdown, and I don't adhere to this schedule strictly; usually however, my day entails the aforementioned activities. I've found so far that it's been an exciting mix of work and play, and I hope to maintain a similar schedule throughout the rest of the summer.

16 Things I'm Grateful For

I think self-reflection is important and gives me the opportunity to be grateful and take on a more positive outlook on life. Today, here are a list of 16 things I'm grateful for in general: Ample food, water, necessities and luxuries Salubrious physical and mental health Wonderful friends who brighten my day A comfortable and fun lifestyle The opportunity to enjoy nature Tasty food The opportunity to work The opportunity to learn Access to an environment conducive to learning The library My dog My brother My parents Happiness The opportunity to relax Familial and social security

Writing about myself

Sometimes, I really don't like writing about myself. To show someone else the kind of person I am, I'd rather just meet them in person. In writing about myself, I am forced to think about how someone else might view me. Writing does not carry the uniquely personal set of emotions, quirks, and genuine responses that can be communicated through speech. I worry about the image of myself that I convey to them, as reflected in my writing, overqualified or underqualified. In writing as such, I communicate not who I am but who I think I am--my self image. It is part of human nature to value others' opinions about you, which frames how you communicate yourself. Thus, writing about oneself is a test of writing ability and self-assessment rather than a true measure of one's personality. I feel more comfortable talking about something other than myself, maybe something that I learned. This draws the attention away from myself, away from the author, and toward a given topic. Most ...

Thoughts on Religion

There are two types of people: those who believe in God, and those who don't. (Well, at least until someone makes this issue non-binary.) Religiosity no doubt has had a significant impact on the lifestyles and morals of people all over the world. I just want to express my thoughts on the nature of religion and its role in society. Over the course of the past few decades, religion has generally gone into decline, along with the rise of science. I think part of the reason is that people now have more reliable, trustworthy answers; they have discovered aspects of life previously thought to be supernatural: how and why planets rotate about the sun, what life is, why people get sick, how to cure disease, what causes lightning, among many others. In addition, people have challenged the nature of religion, which is a sensible thing to do. They take a logical perspective, claiming that there is no scientific evidence that God exists. The opposition to this claim is weak, if even present. ...

Work and fun

This is yesterday's post; I'm publishing today Today, I will share my work and fun life (informally). My name is Ravit Sharma, as you probably know. I was born and raised in a two story house, where I still live. I was brought up being taught I was smart, that I was the best. Looking back, my accomplishments were pretty remarkable. I could read at two and a half, and I knew my multiplication tables till 12 when I was four. These accomplishments make me feel happy; who doesn't like to be praised? But I've realized that there is something more to life than just accomplishments. Working hard gives me a sense of true happiness, more so than does a free A. Today, I feel like I didn't work hard; I can't explain it, but it's just something inside me that nags me, making me feel bad about not working hard. It's almost inexplicable, and doesn't necessarily correlate to the actual number of tasks I finished. It's about the effort I put in, something tha...

Describe yourself in 5 words

Me attempting to describe myself in 5 words. I wrote this yesterday, but am publishing today. Unsure how to describe myself Trying to always learn more Non-conforming Striving to become a thinker Obsessed with Holland as Spiderman Prone to off by one errors Trying to outdo my competition Eagerly awaiting increase in height Typing speed of 80 wpm Current height of 66 inches Current weight of 135 pounds Like relaxing outside at night Goofing around with my brother Learning from my past mistakes An Indian (that's not unique) Pretty cool and rad bro Really bad sense of humor I am a human being Trying to be cool sometimes (and failing miserably) Redundancy in expression of ideas Redundancy in expression of ideas Guided by my moral compass Love when work is fun Unable to think of ideas Want to be a spy Dream of ability to fly Dream of being a superhero Procastinate on tedious but necessary tasks Blind to my own weaknesses Sometimes arrogant and self obses...

Medical Wearables

This evening, I had the chance to attend a medical wearables event hosted by IEEE. There were two presenters, who presented an overview of applications and their involvement in the design and feasibility of such solutions. The field is interesting because it's at the intersection of two orthogonal fields: computer science and biology, more specifically medicine. The first speaker, Don Aoki, presented an accessible solution for consumers for diagnosis of sleep disorders. The solution presented an accessible and convenient product individuals can use for sleep monitoring. The product leverages existing measurement devices, such as EKG, EEG, pulse oximeter, among others, in the assessment of an individual's sleep. These measurements are a subset of the more comprehensive range of measurements made by a polysomnograph made in a clinical setting. However, by making an in-home, affordable solution, the concept bypassed the financial and mental inconveniences of the latter. The solu...

Find x.

Find x. Final answer: x is 7. The beauty of my response is that no one can tell me I'm wrong. If someone challenges my answer, I can just as easily construct a scenario in which x is in fact 7, such as x + 2 = 9. Over the years, I've come to realize that sometimes the solution space is indefinitely large (pardon the linear algebra reference), that there is no one right answer to the problem.  When presented with insufficient information, we have to start somewhere. There isn't always one proper thing to do or one optimal decision to make, but rather a span of actions that satisfy the said criteria, out of which you have to pick one. The idea that we can make choices, each having an uncertain output is thrilling but simultaneously sobering. Ultimately, it is this freedom, this ambiguity, this choice that makes us all unique. In this scenario, such criteria was not laid out; x was not defined. I found x. I made a choice based on incomplete information, and it is one I...

W

The letter w is so weird. So is x, but w is weirder. Why in the world would anyone with a working mind want to call w as "double-u"? For starters, the shape of the letter w doesn't look like two letter 'u's, which is what would be expected given the name. Why not call it "double-v"? Or better yet "upside-down m"? These two would better communicate the shape of the letter. The letter does not fit in with the reset of the alphabet. First, it is the only letter pronounced with more than one syllable. In terms of sound, the letter 'w' doesn't sound anything like two 'u's. Think about it. And why "double"? What is there in 'w' other than the shape? If I said the word "water", it would sound nothing like "uuater" (ooh-ooh-a-tur?) Maybe I should go around calling m as "double-n". Like spell "Mary". Okay so "double-n ... a r y". In addition, the pronunciation of ...

History of Emoji

I really don't want to write a blog post right now, but this is a goal, and I must stick to it :) Since I didn't write a post yesterday, I will write two today. The first is a question I've had for a long time, but one that I remembered yesterday at 11:15 pm while folding my laundry. Okay, so I did my research, and I have a history on the emoji. Originally released in 1997, the emoji has become a universal component of texting, informal messaging, and almost all forms of electronic communication (hey, even this blog has an emoji button). Due to their ability to concisely and conveniently convey emotion, they have risen in popularity, earning their own place on most smartphone and tablet keywords. The origin of the word emoji is from the Japanese words e (picture) and moji (character). Emojis are part of the Unicode Standard and are treated as text, explaining why they may appear differently on different platforms. The second most popular emoji is "Face with Tears of ...

Random Words

Hello everyone! Hope you are doing well. If you are not doing well, I hope you will be doing well soon. I don't know what that means. A couple days back I found an old library book I didn't return. I don't have fines on it, but I will probably return it anyway. Anyhow, I want to try that thing where I just write down the next random word that comes to my head. I will try that for 5 minutes starting now: dog cat elephant zebra africa continent ocean blue globe ball plastic ocean seagull dolphin chirp bird fly wing egg nest twig tree root soil dirt nutrient vitamin amino acid protein peptide carbohydrate sugar sweet milk cow moo sound decibel measurement logarithm exponential e constant variable change delta acceleration differential derivative graph slope hill mountain rocky rocks volcano lava magma core crust sun solar flare space asteroid belt escape velocity projectile motion newton principia mathematica leibnitz calculus integral derivative taylor series polynomial log...

Interview

Following is a (fictitious) interview with Sal Khan from Khanacademy. As in he interviewed me, not the other way around. Sal: Hello. Me: Hi. Sal: Thanks for your time. Me: No worries, it's only $130 an hour. Sal: Tell me a little bit about yourself. Me: My name is Ravit Sharma. Sal: Tell me more about yourself. Me: Okay then. I just woke up about 10 minutes ago and am writing this blog to get it out of the way for the day. Yesterday I learned stuff, so might as well flex my knowledge. So I learned about different algorithms for object detection, including YOLO, RCNN, and Mask RCNN. Let's go through each one at a time. So let's start with YOLO, which stands for You Only Look Once. I'll be discussing YOLO v3 since it is the most recent version. So here's how it works. The image is divided into SxS grid cells. The algorithm takes a sliding window approach to output at each grid cell, an output in the form for each of 3 anchor boxes: the probability of there...

Pineapples

I'm pretty much out of ideas to write about, so why not write about pineapples? Pineapples. P-I-N-E-A-P-P-L-E-S. Why is the word apple in pineapple? They don't look like each other, and definitely don't taste like each other by any means. Oh ok, I see on the dictionary that it was named because of its resemblance to a pine cone. Okay that makes more sense. A pineapple looks more like a pine cone than an apple, at least in terms of texture and maybe color. Pineapple contains bromelain, an enzyme which breaks down proteins. The enzyme bromelain (and papain, found in papaya) are used in meat tenderizer because they are able to break down the extracellular matrix found in animal tissue. This is the same reason that jello with raw pineapples does not set; the gelatin degrades the collagen that makes up the gelatin, preventing the jello from setting (I'm not sure of how exactly jello sets.) But if you denature the enzymes in pineapple by cooking it, it no longer holds the sam...

Update

Third post (yes I realize I missed a day, but I'll write two to make up for the missed one)! So not much has been going on. I finished editing the videos for the Linear Algebra Series and uploaded them onto YouTube. Surprisingly, a video about linear algebra jokes was by far the most popular, with so far almost 200 views. There, I got my first comments. Today, I'll be learning more about object detection and segmentation, and the various ways of doing so quickly for processing data on the edge. It feels so cold at night and in the morning, even though it's summer. I think I'll have to start sleeping with a quilt or at least a sweater. Anyway, today's supposed to be rather warm (but not hot), in the low eighties, so looking forward to a nice summer day. It is a few minutes before 8 am right now, and I woke up around 7:00. I'm on my second week of summer vacation. This year's summer solstice, the longest day of the year, is right around the corner. Although it...

Second Post!

This is my second day streak; my second post this summer. My goal is to continue posting once a day throughout summer. Since I can't seem to think of much interesting to talk about, I'll just talk about what I'm up to this summer. First, I've started a YouTube channel, where I make videos about academic topics. So far I've posted two series and am working on the third. The first one was about local sequence alignment, and the second one was about deep learning. The third one I'm working on right now is about Linear Algebra. The whole process of making a video series typically lasts about three, maybe four days. The first day is planning, where I work out the details of what I'll cover in the series. The next day is filming, where I set up the camera and lighting, and record myself teaching the concepts. The last day is the most tedious, where I have to laboriously go through all the footage, add an intro and ending scene clip, and speed up or cut out some pa...

Random Thoughts

Hello there! As you can probably tell, I've fallen out of the habit of writing blogs regularly. Looking forward, I seek to make blog-writing a regular habit, a tool that I can use to preserve my memories rather than a burden or obligation. On this blog, I'll write about the topics that come to my mind, and spend about 10 minutes a day to pause and reflect. I will not edit this post unless I want to. I think that editing blogs for readability detracts from my experience writing, which is for fun. After all, this blog isn't another English assignment. So let's make it a fun place where I can convey my status, random thoughts, and simply reflect upon my life. (I realize I'm being repetitive, but once again, I don't feel like editing this post.) If you don't like the way I write, don't read it. But once again, I think, "Do I have a standard to hold this blog up to?" At the moment, I don't think so. The purpose of this blog is to capture who I a...

Overview of Reinforcement Learning

What is Reinforcement Learning? Reinforcement learning (RL) is simply a branch of machine learning concerned with optimizing the performance of a software agent on a particular task with experience. It is a field of ML considered distinct from both supervised and unsupervised learning. Markov Decision Processes A Markov Decision Process (MDP) is a model of reinforcement learning consisting of 4 parts: State (S) - information the agent gets from the observation Action (A) - a decision made in a particular state, from a set of discrete or continuous actions Probability that A will lead from one state (S) to another (S') Reward associated with the transition from S to S' Markov Decision Processes are useful in reinforcement learning because they are useful in situations involving the cycle of observing the environment to get a state, and taking an action, which changes the state. Exploration vs Exploitation The issue of exploration vs exploitation refers to the is...