Updated Pico Programming Workflow

Summary: This page introduces the tool chain in the programming workflow for picos. Pico Tool chain and programming workflow
I just got done updating the page in the Pico documentation that talks about the <a href="https://picolabs.atlassian.net/wiki/display/docs/Programming+Workflow">pico programming workflow</a>. I use the idea of a toolchain as an organizing principle. I think it turned out well. If you program picos, it might be of some help. 
Tags:

The New Pico Engine Is Ready for Use

Summary: The new pico engine is nearly feature complete and being used in a wide variety of settings. I'm declaring it ready for use. The mountains, the lake and the cloud
A little over a year ago, I announced that I was <a href="http://www.windley.com/archives/2016/03/rebuilding_krl.shtml">starting a project to rebuild the pico engine</a>. My goal was to improve performance, make it easier to install, and supporting small deployments while retaining the best features of picos, specifically being Internet first. 



Over the past year we've met that goal and I'm quite excited about where we're at. Matthew Wright and Bruce Conrad have reimplemented the pico engine in NodeJS. The new engine is easy to install and quite a bit faster than the old engine. We've already got most of the important features of picos. My students have redone large portions of our supporting code to run on the new engine. As a result, the new engine is sufficiently advanced that I'm <!--more--> it ready for use. 



We've updated the <a href="https://picolabs.atlassian.net/wiki/spaces/docs/pages/19791878/Pico+Engine+Quickstart" >Quickstart</a> and <a href="https://picolabs.atlassian.net/wiki/spaces/docs/pages/1185969/Pico+Programming+Lessons">Pico Programming Lessons</a> to use the new engine. I'm also adding new lessons to help programmers understand the most important features of Picos and KRL.



My Large-Scale Distributed Systems class (CS462) is using the new pico engine for their reactive programming assignments this semester. I've got 40 students going through the pico programming lessons as well as reactive programming labs from the course. The new engine is holding up well. I'm planning to expand it's use in the course this spring. 



Adam Burdett has redone the <a href="http://www.windley.com/archives/2016/07/pico_labs_at_open_west.shtml" >closet demo we showed at OpenWest</a> last summer using the new engine running on a Raspberry Pi. One of the things I didn't like about using the classic pico engine in this scenario was that it made the solution overly reliant on a cloud-based system (the pico engine) and consequently was not robust under network partitioning. If the goal is to keep my machines cool, I don't want them overheating because my network was down. Now the closet controller can run locally with minimal reliance on the wider Internet.



Bruce was able to use the new engine on a <a href="http://www.windley.com/archives/2017/01/using_picos_for_byus_priority_registration.shtml">proof of concept for BYU's priority registration</a>. This was a demonstration of the ability for the engine to scale and handle thousands of picos. The engine running on a laptop was able to handle 44,504 add/drop events in over 8000 separate picos in 35 minutes and 19 seconds. The throughput was 21 registration events per second or 47.6 milliseconds per request.



We've had several lunch and learn sessions with developers inside and outside BYU to introduce the new engine and get feedback. I'm quite pleased with the reception and interactions we've had. I'm looking to expand those now that the lessons are completed and we have had several dozen people work them. If you're interested in attending one, let me know. 

Up Next

I've hired two new students, Joshua Olson and Connor Grimm, to join Adam Burdett and Nick Angell in my lab. We're planning to spend the summer getting Manifold, our pico-based Internet of Things platform, running on the new engine. This will provide a good opportunity to improve the new pico engine and give us a good IoT system for future experiments, supporting our idea around <a href="http://www.windley.com/archives/2015/07/social_things_trustworthy_spaces_and_the_internet_of_things.shtml" >social things</a>.



I'm also contemplating a course on reactive programming with picos on Udemy or something like it. This would be much more focused on practical aspects of reactive programming than my BYU distributed system class. <a href="http://www.windley.com/archives/2015/11/reactive_programming_with_picos.shtml" >Picos are a great way to do reactive programming</a> because they implement an actor model. That's one reason they work so well for the Internet of Things.

Going Further

If you'd like to explore the pico engine and reactive programming with picos, you can start <a href="https://picolabs.atlassian.net/wiki/spaces/docs/pages/19791878/Pico+Engine+Quickstart"> with the Quickstart</a> and move on to the <a href="https://picolabs.atlassian.net/wiki/spaces/docs/pages/1185969/Pico+Programming+Lessons" >pico programming lessons</a>.



We'd also love help with the open source implementation of the pico engine. The <a href="https://github.com/Picolab/node-pico-engine">code is on Github</a> and there's well-maintained set of <a href="https://github.com/Picolab/pico-engine/issues">issues that need to be worked</a>. Bruce is the coordinator of these efforts.



Any questions on picos or using them can be directed to the <a href="http://forum.picolabs.io/">Pico Labs forum</a> and there's a pretty good set of <a href="https://picolabs.atlassian.net/">documentation</a>.

Photo Credit: The mountains, the lake and the cloud from CameliaTWU (CC BY-NC-ND 2.0) Tags:

A System Ruleset for the Pico Engine

Summary: I made a small change to the pico engine recently that has big implications for how we monitor, control, and configure it. FIGURE 10.2 Reinforcing feedback: An increase results in further increase, and vice versa I have a problem: a long time ago, Kynetx built a ruleset management tool called AppBuilder. There are some important rulesets in AppBuilder. I'd like to shut down AppBuilder, but first I need to migrate all the important rulesets to the current ruleset registry. There's just one tiny thing standing in my way: I don't know which rulesets are the important ones. Sure, I could guess and get most of them. Then I'd just wait for things to break to discover the rest. But that's inelegant. My first thought was to write some code to instrument the pico engine. I'd increment a counter each time it loads a ruleset. That way I see what's being used. No guessing. I'd need some way to get stuff in the Continue reading "A System Ruleset for the Pico Engine"

We’re Teaching Programming Wrong

Summary: Event-based programming is more natural for non-programmers than imperative or object-oriented styles. Rule We're teaching programming the wrong way. Interesting study of how people naturally express solutions to problems concludes that starting with imperative programming languages may not be the best way to teach programming skills. What works? Event-based and rule systems, naturally. Maybe we ought to use KRL for introductory programming. I'm willing to try it.
The majority of the statements written by the participants were in a production-rule or event-based style, beginning with words like if or when. However, the raters observed a significant number of statements using other styles, such as constraints, other declarative statements (that were not constraints), and imperative statements. The dominance of rule- or event-based statements suggests that a primarily imperative language may not be the most natural choice. One characteristic of imperative languages is explicit control over program flow. Although imperative languages have Continue reading "We’re Teaching Programming Wrong"

Asynchronous Boundaries and Back Pressure

Summary: Non-blocking back pressume is a useful way to avoid common problems at the asynchronous boundary between autonomous agents. Vanising Train A significant component of reactive programming is asynchronous boundary between the message sender and receiver. The problem is that the receiver might not work as fast as the sender and thus fall behind. When this happens the sender can block (blocking), the event bus can throw messages away (losiness), or the receiver can store messages (unlimited message queue). None of these are ideal. In response, a key concept from reactive systems is non-blocking back pressure. Back pressure allows queues to be bounded. One issues is that back pressure can't be synchronous or you lose all the advantages. Another is that if the sender doesn't have anything else to do (or can't do it easily) then you effectively get blocking. Picos, as implemented by KRL, are lossy. They will queue requests, but Continue reading "Asynchronous Boundaries and Back Pressure"

Errors and Error Handling in KRL

Summary: A small tutorial on how KRL developers can use error events to monitor applications and find problems. fail Errors are events that say "something bad happened." Conveniently, KRL is event-driven. Consequently, using and handling errors in KRL feels natural. Moreover, it is entirely consistent with the rest of the language rather than being something tacked on. Even so, error handling features are not used often enough. This post explores how error events work in KRL and describes how I used them in building Fuse.

Built-In Error Processing

KRL programs run inside a pico and are executed by KRE, the pico engine. KRE automatically raises system:error events when certain problems happen during execution of a ruleset. These events are raised differently than normal explicit events. Rather than being raised on the pico's event bus by default, they are only raised within the current ruleset. Because developers often want to process Continue reading "Errors and Error Handling in KRL"

Picos: Persistent Compute Objects

Summary: This brief introduction to picos and the components that make up the pico ecosystem is designed to make clear the high-level concepts necessary for understanding picos and how they are programmed. Persistent Compute Objects, or picos, are tools for modeling the Internet of Things. A pico represents an entity—something that has a unique identity and a long-lived existence. Picos can represent people, places, things, organizations, and even ideas. The motivation for picos is to design infrastructure to support the Internet of Things that is decentralized, heterarchical, and interoperable. These three characteristics are essential to a workable solution and are sadly lacking in our current implementations. Without these three characteristics, it's impossible to build an Internet of Things that respects people's privacy and independence.

Picos

Picos are:

Picos: Persistent Computer Objects

Summary: This brief introduction to picos and the components that make up the pico ecosystem is designed to make clear the high-level concepts necessary for understanding picos and how they are programmed. Persistent Compute Objects, or picos, are tools for modeling the Internet of Things. A pico represents an entity—something that has a unique identity and a long-lived existence. Picos can represent people, places, things, organizations, and even ideas. The motivation for picos is to design infrastructure to support the Internet of Things that is decentralized, heterarchical, and interoperable. These three characteristics are essential to a workable solution and are sadly lacking in our current implementations. Without these three characteristics, it's impossible to build an Internet of Things that respects people's privacy and independence.

Picos

Picos are:

What’s New With KRL

Summary: In The End of Kynetx and a New Beginning, I described the shutdown of Kynetx and said that the code supporting KRL has been assigned to a thing I created called Pico Labs. Here’s a little more color. Pico In The End of Kynetx and a New Beginning, I described the shutdown of Kynetx and said that the code supporting KRL has been assigned to a thing I created called Pico Labs. Here's a little more color. All of the code that Kynetx developed, including the KRL Rules Engine and the KRL language is open source (and has been for several years). My intention is to continue exploring how KRL and the computational objects that KRL runs in (called picos) can be used to build decentralized solutions to problems in the Internet of Things and related spaces. I have some BYU students helping me work on all this. This past semester they built and released a KRL developer tools application. This application is almost complete in terms of features and will provide a firm foundation for future KRL work. Our plan is to rewrite the CloudOS code and the JavaScript SDK that provides access to it over the summer. The current version of CloudOS is an accretion of stuff from various eras, has lots of inconsistencies, and is missing some significant features. I used CloudOS extensively as part of writing the Fuse API which is based on KRL and picos. I found lots of places we can improve it. I'm anxious to clean it up. As a motivating example for the CloudOS rewrite, we'll redo some of the SquareTag functionality in the PCAA architectural style. SquareTag makes extensive use of picos and will provide a great application of CloudOS for testing and demonstration. I also continue to work (slowly) on a docker instance of KRE so that others can easily run KRE on their own servers. I'm hopeful that these developments will make picos, KRL, and CloudOS easier to use and between Fuse, Guard Tour, and SquareTag we'll have some interesting examples that demonstrate why all of this is relevant to building the Internet of Things. Tags:

Events, Picos, and Microservices

Summary: I spoke at Apistrat 2014 today in Chicago on the Fuse architecture and API. Here are my slides. I spoke at Apistrat 2014 today in Chicago on the Fuse architecture and API. The Fuse architecture is unique because it uses picos and event-query APIs to create a connected car platform. I found microservices to be a useful model for thinking about building features in the Fuse architecture. Here are my slides: Tags:

A Microservice for Healing Subscriptions

Summary: Here is a quick little microservice I wrote as a KRL rule to ensure that the vehicle has all the subscriptions it needs and fix them if need be. The simplicity illstrates why a microservices approach holds such promise for more loosely coupled systems. Such Great Lows Last week I wrote about how Fuse uses a microservices architecture and the benefits that such an architecture provides. This morning I was faced with a problem that the microservices approach solved handily, so I thought I'd write it up as an example. Fuse uses webhooks (we think of them as event channels) to receive notifications that a vehicle has started or stopped, has a low battery, etc. If these subscriptions aren't set up for a vehicle nothing works. Most noticably trips aren't recorded. Alex, who's working on the Fuse app, wasn't seeing any trips from his truck. Sure enough, when I checked, it had no Fuse event subscriptions. I could have just reinitialized Alex's truck, but I figured if it happened to Alex then it's likely to happen to other people, so I created a simple microservice (i.e. KRL rule) that checks the number of subscriptions and if it's lower than the expected number, reinitializes them.
rule check_subscriptions {
  select when fuse subscription_check
  pre {
    vid = carvoyant:vehicle_id(); 
    my_subs = carvoyant:getSubscription(vid);
    should_have = required_subscription_list.length();
  }
  if(my_subs.length() < should_have) then
  {
    send_directive("not enough subscriptions") with
      my_subscriptions = my_subs and
      should_have = should_have
  }
  fired {
    log ">>>> vehicle #{vid} needs subscription check";
    raise fuse event need_initial_carvoyant_subscriptions;
  } else {
    log ">>>> vehicle #{vid} has plenty of subscriptions";
  }
}
The rule uses a pre-existing function to get the current subscriptions and compares the length of that result to the number we should have. If there aren't enough the rule fires and raises the fuse:need_initial_carvoyant_subscriptions. This is a really simple rule. One reason is because it makes use of other services that already exist. There's already a working function for getting the current subscriptions for a vehicle. There's already another service (or rule), called initialize subscriptions, that sets up the subscriptions when they're needed. Another reason the rule is simple is because it doesn't have to figure out which subscriptions are missing and limit itself to only initializing those. If any are missing, it asks for them all. That's because the initialize subscriptions rule is idempotent. You can run it as many times as you like without messing anything up. Of course, I'd rather not put that load on the system if I don't have to, so the check_subscriptions rule checks if something needs to be done before it raises the event. The primary point is that the microservice architecture is loosely coupled and so setting up a service like this is easy. There's very little code, it makes use of other services, and it's unlikely to break anything. I wired it into the system by raising the event it looks for, fuse:subscription_check when the vehicle profile is updated. That seems like a good compromise between over checking and user control. Tags:

Idempotent Services and Guard Rules

Summary: The guard rule pattern provides a way to ensure services are idempotent even when their actions aren't. This post shows how to use guard rules in KRL. Microservices are usually easier to program when responses to an event are idempotent, meaning that they can run multiple times without cumulative effect. Many operations are idempotent (i.e. adding a ruleset to a pico over and over only results in the ruleset being added once). For operations that aren't naturally idempotent, we can make the rule idempotent using the rule's guard condition. Using a guard condition we can ensure the rule only fires when specific conditions are met. Unfortunately, there are some functions in KRL (notably in the PCI and RSM modules) that make state changes (i.e. have persistent effect). These modules are used extensively in CloudOS. When these are used in the rule prelude, they cause side effects before the rule's guard condition is executed. This is a design flaw in KRL that I hope to rectify in a future version of the language. These functions should probably be actions rather than functions so that they only operate after the guard condition is met. In the meantime, a guard rule offers a useful method for assuring idempotency in rules. The basic idea is to create two rules: one that tests a guard condition and one that carries out the rule's real purpose. The guard rule:
  1. responds to the event
  2. tests a condition that ensures idempotence
  3. raises an explicit event in the postlude for which the second rule is listening
For example, in the Fuse system, we want to ensure that each owner has only one fleet. This condition may be relaxed in a future version of the Fuse system, but for now, it seems a reasonable limitation. There are several examples in Fuse where a guard rule is used. The following is the guard rule for the Fuse initialization:
rule kickoff_new_fuse_instance {
  select when fuse need_fleet
  pre {
    fleet_channel = pds:get_item(common:namespace(),"fleet_channel");
  }
  if(fleet_channel.isnull()) then
  {
    send_directive("requesting new Fuse setup");
  }
  fired {
    raise explicit event "need_new_fleet"
      with _api = "sky"
       and fleet = event:attr("fleet") || "My Fleet";
  } else {
    log ">>>>>>>>>>> Fleet channel exists: " + fleet_channel;
    log ">> not creating new fleet ";
  }
}
The guard rule merely looks for a fleet channel (evidence that a fleet already exists) and only continues if the fleet channel is null. The second rule does the real work of creating a fleet pico and initializing it.
rule create_fleet {
  select when explicit need_new_fleet
  pre {
    fleet_name = event:attr("fleet");
    pico = common:factory({"schema": "Fleet", "role": "fleet"}, meta:eci());
    fleet_channel = pico{"authChannel"};
    fleet = {"cid": fleet_channel};
    pico_id = "Owner-fleet-"+ random:uuid();
  }
  if (pico{"authChannel"} neq "none") then
  {
    send_directive("Fleet created") with
      cid = fleet_channel;
    // tell the fleet pico to take care of the rest of the initialization.
    event:send(fleet, "fuse", "fleet_uninitialized") with
      attrs = {"fleet_name": fleet_name,
               "owner_channel": meta:eci(),
               "schema":  "Fleet",
               "_async": 0  //complete this before we try to subscribe below
              };
  }
  fired {
    // put this in our own namespace so we can find it to enforce idempotency
    raise pds event new_data_available 
      with namespace = common:namespace() 
       and keyvalue = "fleet_channel"
       and value = fleet_channel
       and _api = "sky";
    // make it a "pico" in CloudOS eyes
    raise cloudos event picoAttrsSet
      with picoChannel = fleet_channel 
       and picoName = fleet_name
       and picoPhoto = common:fleet_photo 
       and picoId = pico_id
       and _api = "sky";
    // subscribe to the new fleet
    raise cloudos event "subscribe"
      with namespace = common:namespace()
       and  relationship = "Fleet-FleetOwner"
       and  channelName = pico_id
       and  targetChannel = fleet_channel
       and  _api = "sky";
    log ">>> FLEET CHANNEL <<<<";
    log "Pico created for fleet: " + pico.encode();
    raise fuse event new_fleet_initialized;
  } else {
    log "Pico NOT CREATED for fleet";
  }
}
When this rule fires, an action sends an event to the newly created fleet pico that causes it to initialize and three events are raised in the postlude that cause further initialization to take place. Tags: