{code} Represents at the first Dell EMC Public Event!

The Dell EMC Forum in Dallas was an event unlike any other. After all, it was the first united appearance of Dell and EMC as one company. Talk about exciting times!
5311_a4svkc6c9fAs you walked into the Irvine Conference Center, attendees were greeted by a modern setting almost like something out of Star Trek. With over 1,200 registered to attend, the crowd was a mixture of customers, partners and Dell EMC employees many of whom made sure to stop by the {code} booth to try out the virtual reality experience (the HTC Vive paired with Dell Alienware) and talk about open source.

Screen Shot 2016-09-21 at 6.50.10 PM.pngDuring this one day event, Kenny Coleman (Developer Advocate, {code}) gave an introductory session on containers. Kenny  detailed how containers can lay the foundation for the future of infrastructure. He also broke down why one should choose a container platform and how a container is different from virtual machines. With a full room of engaged of attendees, at standing room only, Kenny demonstrated container abilities using Minecraft to forage for inventory as a means of collecting data. He then simulated a server failure which makes the container non-existent but through features within REX-Ray (one of our core projects focused on data persistence for containers) the container was restarted on another host with all of the data intact.

Thank you Dell EMC Forum Dallas team! We look forward to being a part of the Dell EMC Forums in New York City, NY and Long Beach, CA of 2016. We can’t wait to see you there!

Multiplayer Minecraft in Moments

By Akira Wong, Intern from UC Irvine and Proud Heng, Intern from UC San Diego

We learned a ton about how containers impact operating applications over this past summer with the {code} team. Before we began this “Summer of {code},” we needed a goal. We wanted a way to challenge ourselves and it was important that we find a project that was relevant to us and provided opportunities to explore containers, open source and what it means to be a part of the {code} team. After doing some research we came up with our research question:

What would it take to run Minecraft in a container?

Building blocks, adventure, and a playground in one game – Chances are you or someone you know plays Minecraft. To properly grasp this phenomenon check out this short video featured by WIRED.

If you have ever looked into the technical aspects of actually running a Minecraft server, you’ve probably noticed that it is notoriously tedious and even difficult to set up at times. After all, there are numerous hurdles to overcome – to deploy a server one must configure dependencies, open ports, and accept Minecraft’s end user license agreement. While this may seem trivial to some, by the time the rest of us figure out how to maintain and upgrade the server, our friends might have already moved on!

Basically setting up and maintaining a Minecraft server is tough. Players just want to play with their friends and family.

So why not make it an easier process in the long run with containers and persistent storage!

Docker to the rescue!

Docker is a rising star in infrastructure management technology. It’s a tool that allows you to easily deploying applications. We’ll be using what Docker calls containers to run our Minecraft Server. Containers let us create a bundle of everything that’s needed to skip the tedious setup and instead focus on an easy deployment of applications such as Minecraft servers. In addition, containers can be run nearly anywhere – on your laptop, your home server, in a virtual machine or using cloud hosting such as Amazon’s EC2 cloud. Even if you don’t understand the technical details, it’s time to get excited. This means that not only can we deploy a Minecraft server in less than fifteen minutes, it can also stay online forever. If we hand off the configuration to Docker and the hosting to cloud environments, then we have more time to focus on what we actually care about – playing Minecraft.

However, there’s one tiny problem there’s always a hiccup

In exchange for portability and automatic deployment, Docker enforces “statelessness”. This means that a container is meant to live, perform its service, and die.  Ultimately, there is no difference between individual container instances. This poses a major problem to our prospective server – persistence. If  data created by our server will be deleted as soon as the container stops running this questions the sustainability of our Minecraft server.

If you’ve been following along, maybe you already see the looming problem, but for those who don’t quite grasp it…

Imagine you’ve already set up your Minecraft server in a container- it was a breeze, after reading and sharing this blog, you made an AWS account and got a Minecraft server online quickly. Things have been going great. Two weeks have passed, and you and the other players have assembled a castle, built a farm, and established friendly relations with the local villagers. However, you’ve had a nagging feeling in the back of your head lately. What was that thing about containers being stateless? Would our Minecraft server world data be persistent even if our container was killed? What would happen to all our work if this container ever shut down?

As soon as this thought occurs to you, disaster strikes! A janitor trips over a cord, there’s a combination hurricane, tsunami, earthquake, zombie apocalypse and all of AWS goes down. Darkness. Days go by and eventually AWS comes back online. You hesitantly restart the container, trembling as you log in. Only to find…

Everything is gone.

  • That castle? Disappeared.
  • Your watermelons? Never to be eaten.
  • That villager you named Villager McVillageface? You’ll never see him again.

All you can see is the empty expanse of a plains biome. Your players burst into tears when you break the news and you struggle to hold it together yourself. If only this disaster could have been averted. If only there were some way to achieve persistence in containers – then the world may have been saved.

As it turns out there is a way to save state in containers. Using what Docker calls volume mounts it is possible to mount local storage into the container. Then, upon restarting a container with the same volume mount, the data will still be there. However, note that this applies only to local storage, if the container host goes up in flames or you ever want to upgrade it it, then you will need to jump through more hoops.

Fortunately, we can use REX-Ray to circumvent these issues and connect to remote cloud storage volumes. REX-Ray is an open source project from {code} by Dell EMC that allows us to future-proof our Minecraft server. By storing the data remotely, we don’t have to worry if our container or even the whole server is lost while also laying the groundwork to be able to quickly upgrade our server. REX-Ray handles the details of creating storage volumes and connecting one to our Minecraft server container. Using Docker’s volume mounts and REX-Ray we can save our Minecraft world data so there’s no longer need to worry about data being lost forever…

Sound good? Hop over to our GitHub project for detailed instructions and you’ll be up and running in no time.

Building a RackHD home lab

rackhd

RackHD driver for Docker-machine allows a user to provision a Docker instance on bare-metal hardware managed by RackHD. This brings many of the advantages and flexibility of cloud-based infrastructure into your own data center. As major proponents of Infrastructure As Code, we think any tool that makes deploying and scaling infrastructure repeatable and automated is a major win.

One technology that is typically found on the servers in your data center is Out-of-Band Management (OBM), sometimes called “lights-out management.” This is a network connection to the nodes that is independent of their running Operating System (OS) and allows for remote management whether it is power management, OS installation or even remote access to the display. OBM is a foundational tool for RackHD since tasks like remote OS installation and power management are core competencies.

This brings us back to the RackHD driver. While most of the Docker-machine drivers are for virtual cloud-hosted infrastructure, the RackHD driver is unique in targeting bare-metal. While developing the driver, I unsurprisingly found that there was only so much progress I could make using virtualized resources. Not to mention there was code that I couldn’t exercise without bare-metal or OBM. And of course I wanted to use real hardware for testing, but not just any old PC in the closet would do. So, I built a home-lab for RackHD with these capabilities and even managed to avoid breaking the bank.

Here’s how.

My hardware requirements roughly in order of priority were:

  • OBM capable
  • Efficient and quiet (after all, it will sit in my home office)
  • Economical
  • Portable

Performance was not a concern, but getting my manager to okay the expense report was. I ended up getting two nodes, one using IPMI and one using Intel AMT.

Finding a low-cost node with a Baseboard Management Controller (BMC) for IPMI support is surprisingly non-trivial. This is generally an enterprise feature, and therefore exists mostly in servers at enterprise price-points. I ended up finding the SuperMicro MBD-X7SPA-HF-O, a mini-ITX server motherboard. Mini-ITX is a fairly small form factor, and this board included a BMC plus an integrated Intel Atom processor. Add power, RAM, and storage and I was in business! Total cost for chassis, board, hard drive and memory was about $300 from NewEgg.

RackHD also has native support for Intel AMT so I knew I also wanted to look at the Intel NUC nodes. There seems to be only one model of NUC that supports AMT, as it requires vPro support in the processor. That model is the older DC53427HYE, and requires you to supply your own RAM, mSATA drive, and (oddly) a power cord. This includes a Core-i5 processor with vPro and AMT support, and is only about the size of a couple decks of cards. This makes it extremely portable, and potentially great for demos. Hello MeetUps! Total cost for the NUC kit, mSATA drive, memory, and cable was about $475.

Intel NUC.jpg

The NUC cost is higher, but you do get much more powerful hardware and a smaller form factor. I also fitted both nodes with only 4GB RAM and just over 100GB of primary storage to keep costs down.

Both the NUC and SuperMicro node share their OBM Ethernet connection with the same port as their primary Ethernet. This meant I only had one network connection to worry about to each node. I already had all of the cables I needed, and a spare 5-port Gigabit switch on hand to use. I assembled all my hardware, plugged in power and network, and connected the network to a separate wired Ethernet port on my laptop. I fired up RackHD inside of Docker running in Virtualbox using docker-compose, bridging the dedicated wired network port on my laptop to the Virtualbox VM. This allowed RackHD to have full access to the network and serve as the default gateway for DHCP, TFTP, PXE, etc.

RackHD home setup.jpg

I powered on my two nodes and saw…. Nothing. Hmm. Well I suppose it would be a lot to ask for the BIOS of two completely different types of nodes to both be configured exactly the way I would need it.

So I then connected a monitor and USB keyboard, went into the nodes’ BIOS to configure them for PXE booting, and tried again. This time, success! The nodes were discovered and catalogued by RackHD, and I was able to launch OS install workflows against them. Let the docker-machine driver development continue!

rackhdhomelab.jpg

Linux Turns 25 in Style

It’s hard to believe that Linux has already been around for 25 years. What an incredible feat!

Let’s take a moment to go back in time to see the humble beginnings of Linux. Here is the first email from Linus Torvalds on the Minix distribution list, describing what later became Linux as “just a hobby, won’t be big and professional like gnu.

With such a deep history and humble beginnings we couldn’t help but want to celebrate with thousands of passionate people at  LinuxCon and ContainerCon North America 2016! EMC {code} is honored to be a sponsor at this years events. Will you be there?

For those of you planning to attend here are a few things we are looking forward to at the event:

You can find the EMC {code} team at booth #44 in the Expo Hall. We invite you to stop by and learn more about the {code} community and our available open source projects. We look forward to seeing you all and meeting some new faces! Let the countdown begin!


Tux.png

PS: Don’t forget to wish Tux a Happy Birthday! Here’s an idea: why don’t we all break out into song during Linus’ keynote?

Buzz from the Drone Nationals

IMG_1650

One of the obstacles on the race course

It’s difficult to deny the steady rise in popularity of the small, remote-controlled flying devices collectively referred to as “drones”. From hot gift items to the future of home delivery, drones are everywhere! As the industry grows, most of today’s drones flown recreationally rely on open source software. Some of these open source projects have actually caused disruption to the drone industry, since open source allows savvy consumers to customize and extend the capabilities of their drones.

During the first weekend in August, EMC {code} was a major partner at the 2016 National Drone Racing Championships. The races were held on Governors Island in New York City, a beautiful, historic site within view of the Statue of Liberty. Competitors from all over the United States – and some from beyond – raced for the title.

The competition featured events for both multirotor and fixed-wing vehicles. The race course was a challenging series of large fabric-covered hoops erected throughout the outdoor venue. The sky buzzed with the whine of tiny electric motors being pushed to their limits as the speedy drones zipped through the obstacles and past the bleachers full of spectators. The days were hot and humid, with a brief but intense rain shower on Saturday afternoon which halted the races temporarily.

The race course, viewed from atop a nearby hill

The racing drones were controlled using wireless first-person video (FPV) technology; pilots viewed the racecourse through head-mounted visor displays showing a live feed from tiny cameras mounted on the drones. Many fans in the bleachers brought along their own FPV visors, so they could tune into the video feed for their favorite racer and watch along in real time. The tiny vehicles – and their pilots – were pushed to their limits, and several of the drones met sudden and violent ends through spectacular collisions. One competitor, “ZoeFPV” Stumbaugh, even had her custom multirotor drone burst into flames in mid-air when her battery became overheated!

IMG_1668Drones are swiftly emerging as a valuable new market, with some predicting a $12B valuation by the year 2021. Data collection from drones offers a wealth of new and fascinating knowledge, but the sheer volume of that data also presents a challenge for drone operators. Ultimately data storage will be a challenge that companies using drone technology will have to address.

Throughout the weekend the {code} team hosted the Sky Lab – a fun and interactive exhibit containing many stations that visitors could try their hands at piloting drones in virtual reality. Kids and adults alike jumped in to experience the hardware flight controls that flew simulated drones on a virtual copy of the official drone race track. The Sky Lab was so popular that some visitors returned all three days to play the games.

We were very happy to be a part of this exciting event, and to help bring our message of open source community and collaboration to an emerging market. We congratulate all the competitors on their performance, and look forward to the 2016 World Drone Racing Championships coming up in Hawaii this October!

 

Drones Dart Across Governors Island for the Title

Screen Shot 2016-08-03 at 9.25.39 PM.png

We’re stoked to announce EMC {code} was selected as a major partner for the 2016 US National Drone Racing Championship on Governors Island, New York due to our expertise in open source! We join the ranks with GoPro, EY, Vizio, Lowepro and several others in partnership with the Drone Sports Association (DSA). This weekend, from August 5th-August 7th, the EMC {code} team and members of the {code} Community will take over Governors Island along with thousands of drone fans and the nation’s top talent in the racing community.

The Pilots’ path to Nationals was not an easy one. According to the Drone Nationals website, this journey began with “19 qualifiers, 20 teams, 500 pilots and over 101 Freestyle submissions.” The winners of this year’s championship will head to Hawaii for the 2016 World Drone Racing Championships in October. It won’t be easy, but it certainly looks like a lot of fun.

Good Luck to all the pilots and congratulations for making it this far! 

So, bottom line – if you are in New York this weekend, you have no excuse but to hop on a ferry to Governors Island and join EMC {code} in the Sky Lab where we are proud to host a virtual reality and gaming wonderland. The Sky Lab will include three virtual reality headsets (HTC Vive) for real-time experiences and a multi-computer station for attendees to test their flying skills against the pilots’ times on the actual course! If you can’t make it in person, you can follow the action via live stream on ESPN3 this weekend starting at 1PM ET on Saturday, August 6th

General admission is free, be sure to register – we look forward to seeing you on the island!

On a final note, here’s a quick video from the International Drone Racing Association (IDRA) to get you just as pumped as we are…

Connecting EMC, Open Source and Drones! Oh my!

The EMC {code} team is heading to NYC this weekend (Aug 5-7) for the US National Drone Racing Championship. We are thrilled to showcase how open source software meets hardware in a real world environment. In doing so, we asked one of our community members and drone enthusiast from the EMC dotnext Team Matt Cowger if he would share his perspective. Here is what Cowger had to say:

I’m really excited – unreasonably so.

When you get a call from Joshua Bernstein (VP of Technology of ETD for EMC) that you can spend a weekend at the Drone Nationals (the top drone/quadcopter/multirotor racing event in the US) and help represent EMC {code}, is there any other answer but yes?

My inner geek which really isn’t hidden from the world is beyond pumped. So needless to say, I’m really looking forward to working the event and helping judge the competition. As an avid multirotor racer / builder myself, this will be a pretty awesome work trip.

For some, the question has been”what does EMC {code} have to do with drone racing?” Well, the fascinating thing about modern drones is that many of the best ones are based on entirely open source software (OSS), so it’s a perfect area for the EMC {code} team to get involved with.

Let me give you some examples:

  • rmrc-dodo-fc-flight-controller-f3At the heart of every drone is a flight controller. Drones stay in the air by the sheer brute force of propellers pushing air downwards. This means it’s not a naturally stable system and we need constant feedback to keep the system level and in control. The most common system used in racing quadcopters these days is an ARM STMF303 processor running at about 72MHz with a number of onboard sensors (gyroscopes, accelerometers, barometers, etc). These need software to control them and the most popular software to do so is a family of related open source projects, which include Baseflight, Cleanflight, Betaflight and Raceflight. In the spirit of open source, they each focus on specific problems in drone flight to solve, and regularly share solutions and optimizations across the various projects. Each of these projects is hosted on GitHub that provides access to both users and developers so that we can continue to see innovation and advancements to the technology.
    • There are other projects for other kinds of drones too – software for GPS navigation (iNav) and full mission planning with flight optimization (dRonin). Some of these projects have incredibly rapid release cycles (monthly or faster).
  • photo credit AliExpressAlmost as fascinating are the electronic speed controllers, or ESCs. Each motor’s spin rate needs to be precisely controlled between 0 and up to ~40,000 RPM (yes, really!), and it is the job of the speed controller to receive the requests from the flight controller, read the position of the motor and control its spin rate. This needs to happen about 8,000 times per second. As a result, we use systems centered around high performance control units like the Silicon Labs F390. Each of these speed controllers runs its own software, and the most popular is a piece of open source software called BLHeli. There are very frequent updates to this software adding new features from contributions around the world.
  • Lastly, there’s the example of OpenTX.  As if having your flight controller and speed controllers open sourced wasn’t enough… the most common drone transmitter software OpenTX is also open source. OpenTX is incredibly robust and allows you to configure your controller from a laptop interface and run Lua scripts (another open source tool – a scripting language) right there on the radio while you are flying. OpenTX is so powerful and easy to use that it has become the standard in the multicopter industry, with a number of manufacturers forgoing their own firmware and simply using OpenTX.

There are other great examples as well for specific integrated ‘toy’ class helicopter, like the Eachine H8 and Hubsan X4. Both are great small quadcopters to start with.

In short, modern multicopter racing would not be possible without the power of open collaboration tools and open source methods. That’s why the EMC {code} team and I are stoked to support the upcoming Drone Nationals – it’s a glorious combination of fun, geekery and open source.

Will I (we) see you there? – Matt CowgerIMG_2509