My start at Cohere - 26/07/2024

My start at Cohere

In May of 2023, my friends and I decided to attend METHacks at Toronto Metropolitan University. This was my first ever hackathon, and I had no idea what I was doing. The hackathon began as normal with a ton of brainstorming to land on a project idea that was attainable and not completely mediocre. My group landed on Fault Finder, a web app that would use LLM generated code snippets that were intentionally buggy to help train a user on finding and fixing bugs LeetCode style! Cohere happened to be a sponsor at the hackathon and so I took a shot at using their models for generating our faulty code and assertions for testing the users submissions. Initial impressions were not ideal to say the least, at this point in time Coheres models were still in their infancy, and when it came to complex tasks like code generation, OpenAI’s models were the leader.

But still, I saw the vision of the future Cohere was trying to conjure

At a time when OpenAI’s website looked like…this:
Seeing Cohere’s clear thematics and ‘natural’ design language really did push the idea that large language models and humans could mesh together in a way that would irreversibly change the future. I believe the idea of giving technology language is one that will accelerate us to an amazing degree, whether we can ensure that it is done safely and accessible is yet to be determined.

Getting Hired at Cohere

At METHacks, Cohere sent around a posting for a Data Quality Specialist role, I and many others applied and completed their online assessment. After a day, I forgot about the application, I assumed it was in a pile with hundreds of others and would potentially be looked at before its inevitable send off to the recycling bin. Shockingly, just a few days later I received a very warm email from the HR team at Cohere offering me a part time contractual position at their Toronto office to analyse data for their large language models. I hesitated, what about school? Would I really be able to juggle a job in tech and being a fulltime student? I don’t really know anything about AI, am I really cut out for this? What if another opportunity shows up, will I be locked out of it if I accept? These were just a few of the questions that I chewed on while trying to figure out what to do.

Taking a Risk

I eventually came to the conclusion that the only way to grow was to take the risk. With the way that the tech space was accelerating and shifting its focus towards artificial intelligence as not just an area of research, but something that could be made into real products, I recognized that getting in early was the best choice. I started the position on May 23rd of 2023 and was introduced to some of the most amazing managers, coworkers, and machine learning enthusiasts that Toronto has to offer. The learning process was rapid and projects moved quickly, learning how to learn in the workplace was the first big challenge I faced. I quickly realised that the best path to success was to share knowledge. I fondly recall the early days of maintaining rigorous notes about everything and preparing my ideas for how to challenge our models while on my commute to the office.

Of course, there were downsides. Being a full time student and attempting to work 40 hours a week took a hefty toll. Learning how to min-max hours spent studying, working, and sleeping became a careful balancing act that, if disturbed, would throw me off for the day. I don’t regret going through this though. Being able to experience the grind of working towards not just a degree, but also a career through workplace experiences was life changing to put it simply.

I am a strong believer now that the best way to learn is to jump into the deep end, trust me, you’ll make it 💙.

What I did

The main focus of my job was on data creation, auditing, and tracking of model pathologies. Over my time as a DQS I reviewed a few thousand data points by hand, following a great style guide written by an outstanding editor. A lot of my job was subjective, determining what ‘good’ data was could only be guided so closely by instructions. After a while, it did come down to how well I could reason about the pros and cons of any individual data point. This taught me a lot about being able to back up results and opinions, while there were no ‘right’ answers there certainly were wrong ones. The skill of giving reasoning behind my decisions and feedback is something I still use to this day in a more technical role. Alongside this I reviewed code and STEM annotations which required a ton of in domain knowledge. I learned a lot about coding from this. Kind of ironic that a hackathon project about reviewing LLM generated code led me to a job where I reviewed code generated by LLMs. All in all I loved this job, the people were amazing, also the atmosphere and culture in the annotation team was electric and remains so to this day.

After a few months of helping with bigger projects while also helping new hires to learn the ropes and generally helping out coworkers with tricky questions I was offered the position of Senior Data Quality Specialist Lead, where I would be able to lead my own team of data analysts and be at the front of upcoming projects. You can read more about my time as a SDQS in another blog post, it was a huge part of my career.