Keep Growing

Keep Growing - Green plant growing in the glow of technological advancement.

What are your skills? I recently saw some research by IBM that suggests that general skills typically have a “shelf-life” of about 5 years. More critically, technical skills expire in about 2.5 years! Fundamentally that means that for us to stay relevant, we are always re-skilling, learning new things and mutating our skills to meet the fast-evolving landscape of technology. How does that feel to you? Exciting, exhilarating and well, maybe even a bit exhausting? I agree!

Constant change is the very nature of life and all of creation. Life and creative energy strive to grow against the gravity of the status-quo, branching and evolving to become more than it was before. Each generation breaks through the bedrock of normalcy and sends roots into the unknown and absorbs it, learns from it, and flourishes. Our generation is no different. Even today, we are on the verge of a vast shift in terms of technological change and disruption.

But there is a danger. Life yearns to become greater, yet our rival, death is in the shadows, lurking to devour us. Progress demands energy and determination, but it is difficult, and the burden can become heavy, sapping away our enthusiasm. There will be the temptation to stop, to stand still, or to give up. That negative propulsion tugs on us every day. But we have a choice. We can defy gravity. We can send our roots into the unchartered unknown, learn from it and grow. How do we do that?

Stay curious! Move forward, do new things, try new ideas, open new doors. What is current in tech? Explore it and learn about it. A world of endless possibilities lies ahead. We are made for this! To grow, to explore, to experiment, and to enjoy life and all creation. And yes, I know, we are but imperfect dust, stardust to be exact. But that stardust was reborn with a glorious purpose to shine and make a difference, to help each other, to learn, to build, and to grow into what has never been before.

The frontier awaits! Technology doesn’t stand still, it keeps moving, powered by human imagination and energy. Embrace the challenge and keep learning, keep growing.

Imagine 2029

Imagine it is 2029. Describe it to me. What is going on in your life, your family, your job and in the world? What were those amazing things you dreamed about in 2024 and did those dreams come true?

Five years ago, Gene Kim, a good friend and mentor of mine, challenged me to do a thought experiment. He asked me about my goals for 2024. I started off giving general aspirational goals around making a positive impact in the world, at work, supporting my team and my family. He dared me to get specific. He told me to imagine my future self 5 years from now. What would I be proud to have accomplished if it could be anything in the world? Write it down, he said, so you can pull them out in 2024 and compare your dreams to reality. It was so clarifying, so motivating and so powerful.

When I started this exercise in 2019, I confess, I thought some of my goals were audacious if not ridiculous. But as my friend challenged me, I wrote them down anyway.  Here is what I imagined in 2019 about my 2024 future:

  1. My team and I helped Disney deliver a large new guest delighting product that wouldn’t happen without our SRE efforts.
  2. I became an author of a book and several papers on leadership and using DevOps practices to accelerate business.
  3. My team and I partnered with our businesses to expand the impact of SRE and our SRE approach to toil reduction.
  4. I’m an inventor listed on a published patent and have several patents pending.
  5. My family and I enjoyed several vacations together, managed to stay out of debt and were able to fund college.
  6. I’m the maintainer for at least one Open-Source project with over 100 stars and an active community on GitHub.
  7. We updated our home with energy efficient windows, solar panels, and built a domestic robot to help our family at home.

Not all my dreams have come true. But many of them have. In fact, all but one! My robot is still a bunch of lines on a paper, a 3D printed arm and a pile of wires in the garage. Still, I can say with absolute certainty that I wouldn’t have accomplished even half of those things if I hadn’t listened to my wise friend and made note of my goals.

We hunger for a mission and objectives that we believe in. We prioritize time for what we think is important. State your mission. Define your objectives. Make those dreams come true, one step at a time. But it requires that first step… Imagine.

Are you ready? Let’s dream into 2029. Take a breath. Close your eyes and push out all other distractions. What will make you happy? What do you want your world to be like? After imagining that, open Notes or your favorite note taking tool with the title, “Goals for 2029” and write down 7 things. Do you believe in them? If so, save your dreams, revisit them every year or so and make them happen!

Imagination Realm

“The true sign of intelligence is not knowledge, but imagination” – Albert Einstein

What are you going to be for this Halloween? For me, I think I’m going to be… out of candy! I don’t know who keeps munching down our supplies for tomorrow but we may need to make another run to the store.

Last Tuesday, my family and I took some visiting family down to Disneyland. Pumpkins were everywhere! Some of them were even walking around. Little Jedi Padawans, princesses and heroes. I absolutely adore these little ones. A tiny Rey ran up and gave a huge hug to the actual Rey cast member walking across Galaxy’s Edge. A two foot tall Jedi with his light saber, clung on to the leg of Mando struggling to walk through the market with Grogu. At Avengers Campus, a miniature Spider-Man ran with arms open wide towards his hero who had just appeared after flying through the air. The park was full of dreamers, decorated with capes, robes, hats and sabers. I saw Captain Marvel, Tinker Bell, Elsa, Jack, Little Mermaid and so many more…

Tomorrow is Halloween. Kids of all ages will become whoever they want to be. They will wrap themselves up in the wonderful world of imagination and make-believe. They will go on epic adventures, explore new worlds as their favorite character, and if all goes well, pick up some candy.

What will you be? This time of year reminds me of the power of imagination. It unlocks restrictions we place on ourselves and lets us explore alternatives. It would be good to have some of this fantasy magic throughout the year. Try on some new “what ifs” and “why nots” and see if you can gaze into the crystal ball of the future and imagine some new “what can be’s”. We propel ourselves and our human family forward when we step into the imagination realm. Dream it. Do it. The future awaits… and so does some candy.

Have a happy and safe Halloween!

Memory

I have a terrible memory. I get frustrated with myself when I can’t remember someone’s name. Worse, you know those login screens that prompt you for a number they text you? Ideally you should just be able to glance at it and then key in the number, right? Well, I sometimes have to look multiple times to get it right. It’s the same with dates, phone numbers and addresses. It’s embarrassing. I used to say, I have a photographic memory, but I’m always out of film. Sadly, that joke is about to run out of generational memory too.

How is your memory? Do you sometimes get “out of memory” errors when you try to learn something new? You’re not alone. If you are like me, you will find yourself leaning a lot more on notes and digital tools to help “remember.” I have lists for birthdays, groceries, food orders, clothes and gifts. This external memory storage is an incredible blessing. Now I just have to remember where I put the notes.

How do we remember? It turns out that we are made up of tiny little chatty organisms that love to talk to each other. They sit on our shoulders, at the apex of the human structure, behind our smile and the light of our eyes. We have about 100 billion of these little creatures. Their tiny arms reach out and connect with each other. With their dendrites they branch out and listen for incoming chatter from their neighbors. With their long axons arms, they pass along that information, ever the while adjusting that signal through the synaptic contacts. They subtlety change their connections, including adding brand new ones, in response to experiences or learnings, enabling them to form new memories and modify existing ones. Everything we experience through our senses is broken down into signals that are fed into this incredibly complex neighborhood of neurons, listening, adapting and signaling. This is how we remember. Sometimes, I wonder if my friendly neighborhood neurons are on holiday.

Artificial Intelligence seeks to replicate this incredibly complex learning ability through neural networks. Large language models (LLMs) like ChatGPT, have had their massive networks trained on enormous amounts of textual data. Over time, that learning encodes into the digital representation of synaptic connections. Those “weights” are tuned so that given an input prompt signal, the output produces something that matches the desired result. The amount of memory that these can contain is incredible. You can ask questions about history, science, literature, law, technology and much more, and they will be able to answer you. All that knowledge gets compressed into the digital neural network as represented by virtual synaptic weights.

LLMs are often categorized by the number of synaptic “weights” they can adjust to gain this knowledge. They are called parameters. You can run a 7 billion parameter model on your home computer and it will impress you with its vast knowledge and proficiency. It even has a command of multiple human and computer languages. The most impressive models like ChatGPT have 175 billion parameters and far exceed the capability of the smaller ones. It contains the knowledge and ability to pass some of the most advanced and rigorous exams.

Sit down for a minute. I’m going to tell you something that may blow your mind. Guess how many synaptic connections we have sitting on our shoulders? 100 trillion! That’s right, 1000 times greater than the current LLMs that seem to know everything. But that is just the start. Our brain is capable of forming new connections, increasing the number of parameters in real time. Some suggest it could reach over a quadrillion connections. The brain adapts. It grows. It can reorganize and form new synaptic connections in response to our experiences and learning. For example, when you learn a new skill or acquire new knowledge, the brain can create new synaptic connections to store that information. So answer me this, tell me again why I can’t remember my phone number?

Do you understand how amazing you are? I mean, really. You have an incredible ability to learn new skills and store knowledge. If you manage to learn everything your head can store, the brain will grow new storage! This biological wonder that we embody is infinitely capable of onboarding new information, new skill, new knowledge, new wisdom. Think for a minute. What is it that you want to learn? Go learn it! You have the capability. Use it. Practice expanding your brain. Listen. Look. Read. Think. Learn. You are amazing! Don’t forget it!

Dreams

What is your dream?

As a kid I would dream of being a scientist and working in outer space. Like many of my generation, I was inspired by Star Wars. I loved the Jedi and fancied being one myself, but I was absolutely fascinated with spacecraft. I would spend hours in grade school drawing spaceships and orbital space stations while the rest of the class did their lessons. I wasn’t alone. My friends were all enamored with Star Wars and epic adventures. Then I saw the movie TRON. A new passion formed. I wanted a computer so bad I could taste it!

TRON inspired me. I dreamed of creating virtual worlds my programs could live in. I even imagined living in the Grid myself. In fact, I would ride a light cycle all the way to school. To be fair, everyone else just saw an old beat-up BMX bike, but for me, I was fighting for the users. I wrote my first real program in 7th grade. No surprise, it was a space game with flying sprites of rockets, asteroids, and invading aliens. I remember how incredible it felt to deliver that experience and hearing how others were enjoying it. I was a computer astronaut pushing bits around and manipulating the world through code. After college I worked as a civil engineer shaping the physical world through software. I still dreamed of creating fantasy worlds where my love of space, science and technology could collide. Then it happened.

It smelled like dirt and diesel. Large earth moving vehicles were roaring around us. Steel and concrete workers were busy shaping the terrain. We navigated across deep dirt ruts and board covered walkways, eventually making our way to a center area. Tall rock work spires pierced the sky all around us. Then I felt goosebumps. A grin shot across my face as we rounded a corner and suddenly before us was the Millennium Falcon. I couldn’t believe it! The terraforming of Galaxy’s Edge at Disneyland was almost complete. It hit me. I was seeing my dream come true. We had spent the last several years helping craft the software pipelines and systems that would power this adventure. Soon guests of all ages would experience this fantastical journey into the world of Star Wars powered by technology, science, and imagination.

One hundred years ago, Walt Disney had a dream. He dreamed of a company that would inspire and entertain the world through the art of storytelling. It was a vision of a bright and hopeful future. A dream that would cherish and learn from the past but push boldly forward into the future. And it would require the most important thing of all. People. Artist, workers, Imagineers, cast members, and of course, guests. Today, 100 years after Walt and Roy Disney started the company, we keep moving forward, creating new ideas and inspiring others like we were inspired. Every day, we ship encouragement and delight to our fellow humans all over the world. It is what Walt envisioned. It is what we do. We are part of that magic. Walt’s dream continues because of all of you. With a little faith, some trust, and a bit of pixie dust, I’m convinced that we will continue to delight and surprise the world for years to come.

Congratulations, team! Let’s celebrate one hundred years of Disney magic, inspiration, and storytelling… and here’s to 100 years more!

1202

“That’s one small step for man, one giant leap for mankind.” – Neil Armstrong

July 20, 1969. Neil Armstrong and Edwin “Buzz” Aldrin became the first humans to ever set foot on the moon. But it almost didn’t happen and it almost ended in tragedy. As the Apollo 11 Lunar Excursion Module (LEM) was preparing to land on the moon, the onboard navigational computer started flashing a “1202” alarm. The crew had been meticulously following their checklist. Each step, nominal. But now, something was wrong. Abort? As the crew radioed in the situation to mission control, they could feel the adrenaline surge and anxiety rise.

For months, the crew, the nation and the world were anticipating this historic moment. It was one of the most heavily covered and widely watched events in history. An estimated 600 million people were watching worldwide. The mission had captured the imagination of people. Now, all of it was in jeopardy. “1202” alarm! The alarms kept going off. Each time the LEM guidance computer flashed that alarm, it would reboot and restart. Not good! I can almost feel that tension myself. This was a critical stage that would demand precision to guarantee the safe landing of the module on the treacherous moon’s surface below. Sounds like bad news, right? Would this require the mission to abort?

With millions of people, sitting on the edge of their seats, Mission Control finally responded. The mission would proceed. Relief! It turns out that this was a “known error” that NASA had seen many times before during simulation testing. The computer had a capacity of 2KB erasable memory and 16KB of fixed memory. The computer would run several concurrent programs related to navigation, all competing for the limited memory. If a program couldn’t allocate memory, the “1202” alarm would be raised and the system would reboot. At restart, the most important programs would start up again where they left off. Thankfully, the mission would proceed. Neil Armstrong would soon step off of the LEM and millions of people would hear him say those “one small step” historic words.

But the mission wasn’t over. The mission was to get them safely home as well. Unfortunately, while the astronauts were suiting up for their moon walk, they accidentally bumped into the button of a circuit breaker. It broke off. This switch controlled the power running the ascent engine, the one responsible for getting them off of the moon. Unless it could be fixed, they would be stranded on the moon. NASA and US President Nixon were preparing for the worse, drafting speeches to be given when their oxygen supply ran out. Thankfully, it wouldn’t be needed. Mission control didn’t have a solution, but Buzz Aldrin did. His background in mechanical engineering paid off! He looked at the small opening where the circuit breaker had been and realized he could manage to depress the breaker with a small felt-tip marker. He did and it worked! Mission control reported the circuit was closed. In my mind’s eye, I can’t help but play out that scenario. I imagine Buzz pushing in that pen and saying with confidence, “To Infinity and Beyond!”

Problems always happen. It isn’t a matter of “if” but “when”. What do we do to prepare for them? What do we do when they happen? The story above reminds me of the importance of preparation. The “1202” alarm could have killed the mission, but it didn’t because NASA had invested in time to play through the simulation many times. Seeing this exact alarm gave them confidence in the LEM computer’s ability to recover from this condition. Testing is important, not just to prove that something is ready for launch, but to build knowledge. The testing didn’t remove the alert, but gave the mission team a foundation of experience to make difficult decisions in the heat of the moment.

Not every possible condition can be tested or will be discovered during simulation. As the circuit breaker example highlights, creative problem solving is still needed. The Apollo mission is full of stories like this, but it isn’t alone. We need engineers. We need smart creatives who are capable of plotting solutions across seemingly impossible odds.

Hopefully you won’t find yourself stranded on the moon anytime soon, but I bet you could be running simulations for learning or plotting solutions to problems. You are engineers. You are creatives. You are critical to the mission! Thanks for all you do in helping making the impossible, possible, every day.

To infinity and beyond!


References

Images

  • NASA – Aldrin on the LM footpad
    https://history.nasa.gov/ap11ann/kippsphotos/5869.jpg
  • NASA – Aldrin beside solar wind experiment https://history.nasa.gov/ap11ann/kippsphotos/5873.jpg

Chimney Sweeps and Chainsaws

Chimney sweeps and chainsaws. That might be a good prompt for a suspense or thriller story, but that’s not my intention. This last week, as Fall began to arrive, our neighborhood came alive with buzzing, chopping and thudding sounds. Removing dead branches or pruning for safety is important. It improves the health of the tree and maintains the neighborhood aesthetics. At least that is what our HOA says.

Buzz! Roaring chainsaws began screeching their terrifying soprano shrill. Ching! Metal sounds and falling branches could be heard throughout the day. Boom! A palm tree prawn fell to the ground. Pow! Branches drop into a bed of a truck. Swoosh!  Workers drag dead tree carcasses across the lawns. Okay, maybe it is sounding a bit more like a thriller.

As the cooler weather starts punctuating our weeks here in SoCal, we also began to see chimney sweeps showing up. They dance across the rooftops conducting their trade. Soot is removed and chimney caps are repaired. Preparations are underway for the coming winter months. And I can’t help but sing the song Chim Chim Cher-ee

Pruning and preparing. I see a personal application during this season. I suspect some of you, like me, have “dead branches” that need to be removed. Maybe that old meeting series that no longer adds value. It could be that routine or habit that we keep for comfort, but the leaves of value have long since died. Some of the well-worn rituals are rotting away and adding dead weight to our work. It’s time to prune. Clear away the dead prawns and free up your cognitive load. 

Practices, processes and patterns in our lives are helpful and add a warm glow to our days.  But over time, like creosote, they can build up and add the risk of burnout. It’s time to sweep away the soot. Examine the demands on your life.  Look with care at what consumes your energy, your movement, your heart and your mind.  What should you keep? What should you sweep?  Now is a good time to pause and survey the branches.  Clear the flue. Prepare for the winter months. Make room for the new.

And of course, I’m confident as you begin your pruning and sweeping, good luck will ensue!

Chim chiminey
Chim chiminey
Chim chim cher-oo!
Good luck will rub off when
I shakes 'ands with you

  • Sherman, Richard M., and Sherman, Robert B. “Chim Chim Cher-ee.” Mary Poppins, Walt Disney Records, 1964.
  • Images generated using OpenAI’s DALL-E 2 model.

The Unlimited Future

One step to the edge of impossible. And then, further.

“One step to the edge of impossible. And then, further.” – National Geographic

There has been a lot of excitement in the scientific community these last several weeks. First, there is the constant buzz about AI and the pending birth of a real-life artificial general intelligence like Marvel’s fictional J.A.R.V.I.S. (which is just a rather very intelligent system by the way). Then there is this incredible medical news about the experimental anti-cancer drug, Dostarlimab, which had an unprecedented 100% success rate in eliminating tumors. Imagine what that could do for our human family! And now, just this past week, we saw the excitement building over LK-99, a polycrystalline compound that was reported by a team from Korea University to be a room-temperature and ambient pressure superconductor.

The LK-99 news was particularly fascinating to me. And I’m not alone. The scientific community is buzzing about it and excitedly conducting experiments to replicate to confirm or disprove the discovery. One of the things they hope to observe is “flux pinning”. Have you ever heard of flux pinning? Well, I hadn’t, so I decided to check it out. It turns out that flux pinning is a characteristic of superconductors where magnetic flux lines are trapped in place within a material’s lattice structure (quantum vortices). This flux pinning locks the superconductive material within a magnetic field, causing it to levitate. Can you imagine whole worlds built of this material? It may look a lot like Pandora from Avatar! More importantly this leads to benefits like enhanced current-carrying capabilities, higher magnetic field tolerances, and reduced energy losses.

Implications are mind blowing! If a room temperature and ambient pressure superconductor can be fabricated, we could see things like massively reduced losses in power transmission, higher performing electromagnetic devices (e.g. MRIs, motors, generators), revolutionized transportation systems (e.g. maglev trains, lightweight and energy-efficient propulsion systems), faster low-power computing devices and of course, new insights into the fundamental nature of matter and the universe. Of course, LK-99 may not be the superconductor we are looking for, but the quest continues… and we are learning!

I love science! The systematic rigor, the tenacious pursuit of discovery, and the passionate pursuit of understanding our universe is who we are. We thirst for knowledge and hunger for new abilities. It motivates us. It propels us to adapt. It allows us to survive and thrive when conditions are threatening. It is our genius, and perhaps at times, our curse. We are restless and unsatisfied. But that insatiable curiosity compels us to discover, to explore, to test, to add to our knowledge, to create and become more than we were.

Look, I know I’m incurably optimistic to a fault. I know that there are disappointments and failures ahead of us as well. And to be fair, the path to the future can sometimes seem impossible. But oddly enough, it is at those moments that we discover something different and something new. We see, we learn, we step to the edge and we go further! The unlimited future awaits. Let’s go!

One step to the edge of impossible. And then, further.

A Declaration of Happiness

Picture of 4th of July Party generated by AI Stable Diffusion.

“We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness.” – Declaration of Independence of the thirteen United States of America, July 4, 1776.

Bring out the flags, fireworks, hamburgers and watermelons! Tomorrow is Independence Day in the US. I love the phrase “pursuit of happiness. ” It has been bouncing around in my head all weekend. To be fair, I know that happiness can be fleeting. But it is worth pursuing. It requires practice, energy and determination. But the reward set before us is joy, satisfaction and a smile.

I often say that our goal as a team is to help our businesses create great compelling content, products and experiences, better, faster, safer and happier. Those qualifiers are important. That last one, “happier” is especially relevant. Working in entertainment, we ship happiness as a product, but it means more than that. It means creating a work environment that is as delightful and rewarding as the products we produce. I believe happy people produce happy products. In every job that must be done, there is an element of fun! If you don’t believe me, ask Mary Poppins.

It’s true! We can find happiness at work. Some of life’s greatest moments are at the other end of hard work and effort. There is a great happiness that can be found in a job well done and a mission accomplished. For those of us in technology, solving complex problems and engaging creative energy to design and produce software, systems and automation is both taxing and rewarding. The beautiful paradox is that those things that are the most difficult, challenging and fraught with anguish, are often the ones with the highest dose of happiness, delight and satisfying accomplishment.

How are we doing? Are we meeting our mission in making this a happy place to work? Last week I sent out an end of quarter “happiness survey” to my team (with a scale from 1, “not happy at all” to 5, “extremely happy”). The last several months have presented us with some difficult challenges. I very much wanted to hear from all of my team to see how we were doing. The results were encouraging and informative. They highlighted some of the good but also the challenges before us and areas we need to improve.

Candid feedback is pure gold! Leaders need clear signal. My team impressed me with their candor. I was delighted. I assembled my leadership team and we looked at every comment and will be taking action to follow up on every suggestion and concern raised. It is a priority for us, as it should be.

Now, for your part…. You matter. Your happiness matters. Life is what you choose to make it. What can you do to cultivate happiness for yourself? Today, I challenge you all to think about how you are pursuing happiness. What does it look like for you? What can you change to make it better? Find your path to happiness. Look for it in your work, pursue it with all your heart and enjoy it. I wish you happiness this week and always!

For all of you in the USA, Happy Independence Day!

4th of July Party Image with hamburger, watermelon, US flags and fireworks.

Top blog post image of “4th of July Party” generated by Stable Diffusion AI and second image by Dall-E AI.

JasonGPT-1 : Adventures in AI

Distorted sci-fi black and blue world.

“Imperfect things with a positive ingredient can become a positive difference.” – JasonGPT

I don’t know how you are wired, but for me, I become intoxicated with new technology. I have a compulsive need to learn all about it. I’m also a kinesthetic learner which means I need to be hands on. So into the code I go. My latest fixation is large language models (LLMs) and the underlying generative neural network (NN) transformers (GPTs) that power them. I confess, the last time I built a NN, we were trying to read George H.W. Bush’s lips. And no, that experiment didn’t work out too well for us… or for him! 

Do you want to know what I have discovered so far? Too bad. I thought I would take you along for the ride anyway. Seriously, if you are fed up with all the artificial intelligence news and additives, you can stop now and go about your week. I won’t mind. Otherwise, hang on, I’m going to take you on an Indiana Jones style adventure through GPT! Just don’t look into the eyes of the idol… that could be dangerous, very dangerous!

Where do we start? YouTube of course! I have a new nerd crush. His name is Andrej Karpathy. He is a Slovak-Canadian computer scientist who served as the director of artificial intelligence and Autopilot Vision at Tesla and currently works for OpenAI. He lectured at Standford University and has several good instructional lectures on YouTube. I first saw him at the Microsoft Build conference where he gave a keynote on ChatGPT but what blew me away was his talk, “Let’s build GPT: from scratch, in code, spelled out.” (YouTube link). It’s no joke. He builds a GPT model on the works of Shakespeare (1MB), from scratch. After spending nearly 2 hours with him, Google Colab and PyTorch, I was left with a headache and some cuts and bruises. But I also had an insatiable desire to learn more. I have a long way to go. 

The way I learn is to fork away from just repeating what an instructor says and start adding my own challenges. I had an idea. I have done a lot of writing (many of you are victims to that) and much of that is on my blog site. What if I built a GPT based solely on the corpus of all my writing? Does that sound narcissistic a bit to you too? Oh well, for the good of science, we go in! Cue the Indy music. I extracted the text (468k). It’s not much, but why not? 

By the way, if you are still with me, I’ll try to go faster. You won’t want to hear about how I wasted so much time trying to use AMD GPUs (their ROCm software sucks, traveler beware), switched to CPUs, Nvidia CUDA and eventually Apple Silicon MPS (Metal Performance Shaders built in to the M1). All the while, I was using my fork of the code I built with Andrej Karpathy (ok, not him directly, but while watching his video). I started off with the simple Bigram NN Language model. And it is “Bi-Gram” not “Big RAM” but I found that to be ironically comical in a dad joke sort of way. 

My JasonGPT bigram.py started learning. It ran for 50,000 iterations and took about 8 hours. It even produced an output of random musings. While there was quite a bit of nonsensical output, I was amazed at how well this small run did at learning words, basic sentence structure and even picked up on my style. Here are some samples from the output I found interesting, comical and sometimes, well, spot on:

  • It’s a lot of time… But I think we also need science.
  • What are your big ideas?
  • Set our management to the adjacent ground (GND) pin.
  • I have a task to Disneyland out that this day.
  • I love the fun and fanciful moments as kids get to dream into their favorite characters, embrace the identity of their heroes, wrap themselves up starfish back.
  • Bring on the “power” of his accidental detail.
  • Your character provided faith, all kindness and don’t care.
  • Grab a difference too.
  • After several days of emailing, texting and calling, I received a text message.
  • Curl has the ability to provide timing data for DNS lookup, it will easily show or avoided.
  • Imperfect things with a positive ingredient can become a positive difference, just get that time.
  • I also believe we should exploit the fusion power that shows up each day in our company’s data.
  • Have you found a vulnerability? Are you concerned about some missing measures or designs that should be modernized or addressed? If so, don’t wait, raise those issues. Speak up and act. You can make a difference.
  • “I know what you are thinking.” the irony
  • We are the ones who make a brighter day.
  • The journey ahead is ahead.
  • What are you penning today? What adventures are you crafting by your doing? Get up, get moving… keep writing.

Look, it’s no ChatGPT, but it blew my mind! I’m only using a 4 layer NN with 7 million parameters. In comparison, ChatGPT uses 96 layers and 175 billion parameters! Before the weekend ended, I set up nanoGPT to build a more elaborate model on my data set. It’s still running, but already I can see it has learned a lot more of my style but seems to lack some focus on topics. It’s easily distracted and interrupts its own train of thoughts with new ideas. Squirrel! Nothing like me.

So my JasonGPT won’t be writing my Monday updates anytime soon, but who knows, maybe it will help me come up with some new ideas. I just hope it stays benevolent and kind. I would hate for it to suddenly become self-aware and start…

Connection to imac.local closed.