September 6, 2016 2:10 pm

Windows 10 IoT Core Blockly

By / Dev Lead, IoT Core Developer Experience

In this blog post you’ll learn about IoT Core Blockly, a new UWP application that allows you to program a Raspberry Pi 2 or 3 and a Raspberry Pi Sense Hat using a “block” editor from your browser:

IMAGE1

You create a program with interlocking blocks, which will run on the Raspberry Pi. For example, you can control the LED matrix on the Sense Hat and react to the inputs (buttons, sensors like temperature, humidity, etc.).

IMAGE2

IoT Core Blockly was inspired by these other super interesting projects:

In this blog post, we will show you how to set up your Raspberry Pi with IoT Core Blockly and get coding with blocks.

Also, we’ll open the hood and look at how we implemented IoT Core Blockly leveraging Windows 10 IoT Core, Google Blockly, the Sense Hat library and the Chakra JavaScript engine.

Set up IoT Core Blockly on your Raspberry Pi

What you will need:

  • A Raspberry Pi 2 or 3
  • A Raspberry Pi Sense Hat
  • A clean SD Card (at least 8 Gb) to install Windows IoT Core 10 Insider Preview
  • A computer or laptop running Windows 10, to install the IoT Dashboard

First, unpack your Sense Hat and connect it on top of the Raspberry Pi (having four small spacers is handy, but not required):

IMAGE3

IMAGE4

Now you will need to install the latest Windows IoT Core 10 Insider Preview on your SD card. Follow the instructions at www.windowsondevices.com in the “Get Started” section:

  • If you have a Raspberry Pi 2, go here
  • If you have a Raspberry Pi 3, go here

At this point, you should have the IoT Dashboard up and running on your Windows 10 desktop (or laptop) and your Raspberry Pi connected to the network (either Ethernet or Wireless). You should be able to see your Raspberry Pi on the IoT Dashboard “My devices” section:

IMAGE5

In IoT Dashboard, go to the section called “Try some samples.” You will see the IoT Core Blockly sample:

IMAGE6

Click on it and follow the instructions to install the UWP application onto your Raspberry Pi. After a few seconds, IoT Dashboard will open your favorite browser and connect to the IoT Core Blockly application running on your Raspberry Pi:

IMAGE7

Press the “Run” button and the IoT Core Blockly application will start the “Heartbeat” program, and you should see a blinking red heart on your Sense Hat!

Try some other samples (the green buttons on the top). Select a sample, inspect the “blocks” in the editor and press the “Start” button to start this new program.

Try modifying an example: maybe a different image, color or message. IoT Core Blockly remembers the last program you run on the Raspberry Pi and will reload it when you start the Raspberry Pi again.

Under the hood

How does IoT Core Blockly work? How did we build it?

The code is on GitHub: https://github.com/ms-iot/samples/tree/develop/IoTBlockly.

You can clone the https://github.com/ms-iot/samples repo and load the IoTBlockly solution using Visual Studio 2015 Update 3.

The structure of IoT Core Blockly is simple:

  • The main app starts a web server which serves the Blockly editor page on port 8000.
  • At this point, you can browse to your Raspberry Pi <ip-address>:8000 from a browser and access the Blockly editor.
  • We created custom blocks for specific Sense Hat functionalities (e.g. the LED display, the joystick, the sensors, etc.) and added them to specific “categories” (e.g. Basic, Input, LED, Images, Pin, etc.)
  • Blockly makes it simple to translate blocks to JavaScript, so we could generate a runnable JavaScript snippet. You can see what your block program translates to in JavaScript by pressing the blue button “Convert to JavaScript” – note: to enable “events” like “on joystick button pressed” we have a few helper JavaScript functions and we pay special attention to the order of the various functions.
  • At this point, we have a block editor that can generate a runnable JavaScript snippet: We need something that can execute this JavaScript snippet on a different thread without interfering with the web server.
  • To run the snippet, we instantiate the Chakra JavaScript engine (which is part of every Windows 10 edition) and start the snippet. Chakra makes it easy to stop the snippet at will.
  • Many of the blocks interact directly with the Sense Hat. We could have written a bunch of JavaScript code to control the Sense Hat, but we leveraged the complete and easy to use C# SenseHat library from EmmellSoft. Bridging between JavaScript and C# was extremely easy leveraging a wrapper UWP library.
  • Last, we added some machinery to make sure the last “run” snippet is saved on the Raspberry Pi (both the blocks layout and the JavaScript snippet are cached) and run again the next time the IoT Core Blockly app starts (e.g. when you restart your device).

If you inspect the IoTBlockly solution from GitHub, you can see 4 different projects:

  • IoTBlocklyBackgroundApp: The main app that orchestrate the web server and the Chakra JavaScript engine. Google Blockly code is part of this project.
  • ChakraHost: A wrapper library to simplify the use of the Chakra JavaScript engine (inspired by the “JavaScript Runtime Sample” you can find at https://github.com/Microsoft/Chakra-Samples (full path at http://bit.ly/1QgvdWa).
  • IoTBlocklyHelper: A simple UWP wrapper library to bridge between C# code and the JavaScript snippet. The SenseHat library from EmmellSoft is referenced in this project.
  • SimpleWebServer: A rudimentary web server based on Windows.Networking.Sockets.StreamSocketListener.

Let us know if you want more details about how this works. We can definitely geek out more about this! J

Next steps

IoT Core Blockly is a functional sample, but we think we can do more. For example:

  • add more support for colors,
  • add support for sounds,
  • add support for voice (speech and speech recognition),
  • add more blocks for GPIO, ADC, PWM, SPI, I2C,
  • add blocks for other “hats” (for example, the GrovePi),
  • send/Receive data to/from the cloud, over BT, etc.,
  • leverage the io environment,
  • improve the stability of the web server.

First, though, we want to hear from you J How you use IoT Core? What would you change or improve?

Don’t forget, IoT Core Blockly code is here on GitHub, and we gladly accept help, contributions and ideas!

Thanks for trying this out, and get started with Visual Studio!

The Windows team would love to hear your feedback. Please keep the feedback coming using our Windows Developer UserVoice site. If you have a direct bug, please use the Windows Feedback tool built directly into Windows 10.

Updated September 13, 2016 1:39 pm

Join the conversation

    • Hi Marco,
      Actually, you can use JavaScript to write UWP apps, and Microsoft is very active in the JavaScript space (for example, check out the efforts with the new ECMAScript language improvements, or the work done on TypeScript).
      Specifically for this project, we’re using an interesting mix of JavaScript and C#. We need a “script” language because we need to generate code on the fly as you lay down the code blocks… generating C# and then trying to run it would not work as well (you would need to compile it, etc.)
      Note that the JavaScript easily calls into a C# WinRT library to interface with the PiSense.

      Hope this clarify our approach.

      Thanks,
      Ale Contenti
      Dev Lead, IoT Core Developer Experience

      • Yes it is very apparent that Microsoft is investing into JavaScript, maybe perhaps too much so, leading to scenarios exactly such as this. Introducing languages that are not .NET is incredibly disruptive and sends a very confusing message to .NET developers. It is one thing to offer C# support AND JS support, and another altogether to offer solely JS, as is the case here. Clearly no one there in your group understands that by introducing an incompatible language such as JS into a .NET-based solution, you have forced .NET developers to learn and utilize TWO languages, which leads to TWO code bases, which leads to TWICE the development cost in development, management and BUGS.

        Very terrible to see this terrible strategy and mindset employed by your organization. Do you support .NET or not? If so, support it FIRST and then onboard other ubiquitous languages such as JavaScript. I understand JS’s appeal from an onboarding and adoption perspective, but you continue to employ strategies where JS is the sole language that can be used without a .NET counterpart and it is obviously incredibly frustrating, as well as confusing.

        As for C# not compiling on the fly, I thought this is what the whole .NET Core initiative was all about?

        • Hi Marco,
          I’m a bit confused. Are you saying that C# is not supported well on Windows 10 devices? I’d argue the opposite… We have very good support for C#, XAML, etc.
          Specifically for Windows 10 IoT Core, check out the samples at http://www.windowsondevices.com... lots of C# examples there 🙂

          HTH,
          Ale Contenti
          Dev Lead, IoT Core Developer Experience

          • No not what I am saying at all. Yeah, you support “Xaml” but the API that has been in WinRT/UWP is a joke compared to WPF, and you simply do not care to update to make Windows development fun again. After what, 4 years now? Where is the passion? No one cares there.

            What I am saying is that C# should be fully supported in every scenario and app you create in addition JavaScript (or other languages). Doing otherwise sends a very poor message, and reflects poorly on your group.

            Also, as for making C# work in the browser. In addition to using .NET Core, which was designed from the ground up to compile on the fly, why not actually show some innovation and make that scenario work? Rather than blindly adopt and support an incompatible — and ultimately competing — language to .NET? Here’s an example from some developer who shows a little more ingenuity towards your technology than you do: http://jsil.org/try/

            It is outstanding that your group and MSFT leadership continues to prescribe and embrace technologies that are incompatible with .NET, and by proxy dramatically increase our development and maintenance costs. Truly a hallmark of incompetence and gross apathy.

            This isn’t a rant against you and your IoT group specifically, Ale, but more of the Windows Group as a whole, which continues to disappoint, when the rest of Microsoft continues to excel and improve in their respective areas.

    • +1! 10000% agree with this!! Why does Microsoft continue to introduce and promote non-.NET languages into .NET solutions? Do you not see how this impacts our overhead? With .NET we simply develop with one language and all is great. Bringing JS completely ruins this and is something else we have to learn and account for in our development processes. Not to mention it is a terrible language when compared to .NET! Might as well be COBOL!!

      Please, listen to the 1000s of developers demanding innovative change:
      https://visualstudio.uservoice.com/forums/121579-visual-studio-2015/suggestions/10027638-create-a-ubiquitous-net-client-application-develo

    • +1 to no JavaScript. Truly embarrassing and confusing to see Microsoft continue to support non-.NET languages, especially when so many developers are asking for exactly not that. JavaScript aside, it would be nice to see improvement to the Xaml system and make it more like WPF, too. Everything else looks really great, however!!

  1. Looks great, could get the non-devs on our team to toy with the Pi.

    BTW, I get why you use JavaScript over C# – right tool for the job. C# isn’t appropriate for every use case. These other guys evidently don’t get that.

    • You must be a manager or someone with a very narrow sense of technology. People who say “right tool for the job” and other such dated cliches are generally interns or someone who will be in another industry in the next year or so.

      JavaScript is a scourge to .NET and it does indeed present a problem to the developers who have invested their years into it. When you introduce it or any other language such as python or ruby into your .NET environment you have increased the amount of liability and costs associated with your workflow with each and every introduced language.

      The ignorance here is staggering and obviously permeates the entire Microsoft leadership, who continue to ignore the incredible chorus of developers asking for a more unified, effective, and efficient ecosystem, while weeding out the costly, incompatible, and redundant deficiencies, the biggest easily being JavaScript and the current web application “guidance.”

      The referenced link above has nearly 5,000 votes towards it, and there are other now-closed votes that have been ignored by Microsoft which carry much more. Example:
      https://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/3556619-silverlight-6

      Read the comments for a good bit of entertainment and a lesson in corporate customer relations failure. All of these ask for the same thing: a .NET that works in a browser and is compatible with past and existing investments, reducing costs and overhead.

      Conversely, how many votes can you find that say JavaScript is a good idea and is desired by Microsoft/.NET developers? You are on the wrong side of the developer tide, my friend.

      • Nah, heads-down developer/software engineer with 12 years’ experience. I have a broad understanding of the .NET ecosystem, from ASP.NET to WPF to UWP. I mostly program in C#, and I love the language.

        But it is not suitable for every problem. There is no single language or platform that is. I hate JavaScript as a language, but it’s powerful because it’s standardized. It’s also very easy to manipulate dynamically, which is, I assume, why the team chose it here. I use JavaScript sparingly, but sometimes it’s the right tool.

        You say: “The referenced link above has nearly 5,000 votes towards it…” So? The idea is nice in theory, but 5,000 people CAN be wrong.

        You say: “All of these ask for the same thing: a .NET that works in a browser and is compatible with past and existing investments, reducing costs and overhead.” Yeah, that was already invented. It’s called Silverlight. How’d that work out?

        You say: “Conversely, how many votes can you find that say JavaScript is a good idea and is desired by Microsoft/.NET developers? You are on the wrong side of the developer tide, my friend.” And yet the whole world is embracing HTML5/JavaScript all around us. That’s the developer tide, and if a dogmatic .NET developer won’t recognize that, he’s gonna be obsolete pretty darn soon.

        In any event, I can’t understand the opposition here – this is a tool that dynamically generates JavaScript. You don’t have to edit it or even understand it. If you need more complex functionality than this tool can provide, you can always use C# to program your IoT device.

        • I am one of those 5,000 people (just voted), and even 15,000 people (loved Silverlight!) and I can say for certain that your thinking is exactly the type of nonsense that has led us into the situation we have now. Microsoft got it wrong with Silverlight, sure, but that doesn’t mean that .NET cannot work in the browser. Did you not try out that JSIL link? That is the product of one genius kid developer as a side project. It also seems like what an entire Microsoft division should have done rather than embracing an incompatible and incongruent development paradigm like HTML5/JavaScript and dance on stage like a couple of idiots during Mix’2011 while they were trying to impress all the developers in a deathly quiet room with moving sprites in a web page.

          For having over a decade of experience with a technology you certainly seem short-sighted and not very keen on innovation with the language you supposedly enjoy. You also seem to be missing the point that by context switching between two different languages, you are spreading your time and therefore learned knowledge between the two, which ultimately dilutes your quality (and value). Plus now you have two codebases, each with their set of bugs to manaage. Do you really enjoy this? Again, if you are seasoned you would know and acknowledge that this is undesirable.

          You are right, though, the world IS bracing JavaScript/HTML — even Microsoft, which is the fundamental problem .NET is facing. Companies are ditching .NET because they can use JavaScript everywhere. Just look at NodeJS. Why use .NET when JavaScript works just as good and works everywhere?

          But rather than innovate like the JSIL link above, Microsoft are sitting on their hands and simply making things worse.

          Also, there are projects such as https://facebook.github.io/react-native/ which clearly challenge your notion that a well-built/designed language can indeed be the right tool, all the time. Again, NodeJS is also a good example as well. These are the sorts of innovations that Microsoft should be creating, and that their developers are asking for, but alas, we are stuck with short-sighted thinking and attitudes such as yours. I really hope you can see what everyone is asking for and continue to make .NET relevant for a long, long time!

          • “Do you really enjoy this?” Yeah, I kinda do. I’m not afraid of learning, change, or adaptation. I like having multiple marketable skills.

            What you’re arguing for is a beautiful vision. I think Microsoft had the same vision with Silverlight and now with Xamarin. But, if it’s possible to do, it will take YEARS, more than just a couple, to pull off. The technical challenges are gigantic. What do you propose we do in the meantime? Stomp our feet and refuse to provide solutions because we don’t like JavaScript?

            Rumsfeld: “You go to war with the army you have, not the army you might want.” You solve a problem with the tools you have. They chose JavaScript for this product because it’s the tool they have that accomplishes the goal.

            Don’t like it? Vote with your feet! Ignore Blockly.

          • I agree with Michael in that learning other languages is a good way to see how others think and going about solving problems. Definite value there. However, when it comes to longer term investment like you have with .NET going on for over 15 years now, the benefits on leveraging existing investments cannot be underscored enough. The way I look at it is new language investigation is great for prototyping and small projects. Framework approaches like .NET and Java are for the real-deal, long-term projects. These are where you want to maximize risk and costs wherever and as much as possible.

            Michael, it would indeed take YEARS to accomplish what everyone is asking for here, but part of the point here is that we have had YEARS — about 5 of them — since Microsoft made the ill-fated decision to embrace HTML5/JS as a “complimentary” approach to developing applications along side .NET. I put that in quotes because as previously explained it is nothing but that. Instead of wasting our time during those five years, how much better off would we all be now if Microsoft had done the intelligent, innovative (and brave) thing and made .NET available like JSIL mentioned here?

            Fret not, however, WebAssembly is right around the corner, and from the sounds of the Xamarin folks, due to the work that they have recently done with their iOS magic, it should be relatively simple to incorporate it:

            https://www.reddit.com/r/programmerchat/comments/4dxpcp/i_am_miguel_de_icaza_i_started_xamarin_mono_gnome/d1v9xyd

            Finally, I wouldn’t ignore Blockly. Outside the incorporation of dreaded JavaScript, it looks pretty cool. This goes for the rest of the IoT initiative from Microsoft. 🙂

        • Wow, someone who looks at 20,000 people and say that they ALL could be wrong, and then says that developing two separate, incompatible code bases — each with their own set of defects and cost considerations — is a good idea. This is even after Microsoft acquired Xamarin to fix this exact problem in iOS and Android.

          Sounds like you have found the perfect candidate for your “management” or “leadership” team, Ale Contenti!

          • Straw man argument. Our C# and JavaScript are part of the same code base. They work together to solve problems. There is no duplication between them.

            The logical extension of your argument is that we should abandon WPF because, hey, couldn’t you write a user interface in C#? Let’s abandon SQL, because… C#! Why work with R or F# or C++? We have C#! It’s the underpants gnomes approach to software development. If I need to manipulate the objects in a browser DOM, I need JavaScript. C# won’t do that. It never will.

            I wouldn’t want to work for Microsoft. I’d be a cog in the machine; I much prefer to be the master of my [small] domain.

          • > “Our C# and JavaScript are part of the same code base. They work together to solve problems. There is no duplication between them.”

            This is really laughable. Let’s start with a simple example such as logging. In .NET/C# you might have a logging framework such as Serilog. Now, how do you bring that over into JavaScript? There isn’t really an official and supported version of it in JavaScript. There’s an UNOFFICIAL version of it, but it is totally out of sync and has nowhere near the feature parity of the current .NET version.

            So, you must use another logging solution. Or you can even brave forward with the unofficial repo, it really doesn’t matter. What matters is that the artifact that you ultimately use to define the logging in your JavaScript-based application is written in a language that is not possible to leverage and reference in a .NET solution.

            So let’s say we take a simple Serilog sink. You now have one in .NET, and you now have one in JavaScript. One is completely independent of the other, and both now have their own code, one written in .NET and the other in JavaScript. They are conceptually the same logging sink, but both are now written in incompatible languages. Bugs that occur and are fixed in one language, may or may not have to be fixed in another.

            This is what is meant by duplication, and to be more technical is breaking encapsulation and DRY: https://en.wikipedia.org/wiki/Don%27t_repeat_yourself

            And we’re only talking about logging here. This also obviously extends to IoC/ServiceLocation, exception handling, UI, etc. etc. You lead to a JavaScript counterpart for every .NET part. Which means twice the code to develop and ultimately twice the headache to manage and maintain.

            > “The logical extension of your argument is that we should abandon WPF because, hey, couldn’t you write a user interface in C#?”

            WPF is a .NET user interface framework, which allows you to (programatically) interface with it using C#. So I am not following your “logical” extension. The more natural extension is the cost-prohibitive nightmare that you are living: we have to abandon WPF because it cannot work in a browser, so you must use HTML5/JavaScript instead.

            > Let’s abandon SQL, because… C#!

            We already do. See EntityFramework and LINQ. And it is not C#, it is .NET, which means you can write it in C#, VB.NET, and even F# if you choose.

            > Why work with R or F# or C++? We have C#!

            Two of these languages are already .NET, and I would not be surprised to see R become .NET in the future, but you are exactly on the ball and proving my point precisely. It’s not C# that provides the core, inherent value, but .NET. If the language the developer writes is in .NET, then it can leverage all prior investments and integrate with existing .NET libraries, written in a .NET language which may or may not be C#. That is exactly the point of the cost-savings and quality improvement by utilizing such a paradigm, and exactly why incorporating a paradigm that is incompatible with it (HTML5/JavaScript) is so prohibitive and costly.

            > I wouldn’t want to work for Microsoft. I’d be a cog in the machine; I much prefer to be the master of my [small] domain.

            Now you are finally starting to make sense, my friend. 🙂