Is REST tired? GraphQL a stimulant?

Based on my quick review, it seems a company should add GraphQL to their tech stack. It’s not an either/or situation.

REST
Representational State Transfer (REST) is the architecture underlying the WWW. It scales. The REST concept is also be used to create RESTful services as an alternative to older technologies such as Remote Procedure Calls (RPC) and SOAP. REST architecture scales.

REST has some limitations and critiques. For example, few REST APIs are truly “RESTful”. The level of REST is captured in the Richardson Maturity Model (RMM). The highest level requires use of HATEOAS, see “Designing a A True REST State Machine”.

“A very strong argument could be made that if most APIs are REST-ish instead of REST-ful, and assuming that most of the conventions that we’re actually using boil down to making URLs consistent and basic CRUD, then just maybe REST really isn’t buying us all that much.” — @brandur

One claim is that REST has issues since it is misapplied:
“…“REST is intended for long-lived network-based applications that span multiple organizations” according to its inventor. This is not a requirement for APIs that serve a client app built within the same organization.” — https://reactjs.org/blog/2015/05/01/graphql-introduction.html

GraphQL
GraphQL was created to fill a hole in modern web API. GraphQL unlike common REST queries, describes the ‘shape’ of data to retrieve from a single end-point. The structure of that data at the end point is a black box. The host knows how to fill in the shape to create the response. Kind of like SQL (in a very rough kind of comparison): In SQL the data has a shape, the Relational Model, and the single end-point queries, declaratively describe what to get from that shape. In REST when you ask for a cup you get the kitchen sink and all the cabinets.

“GraphQL is a declarative data fetching specification and query language for APIs. It was created by Facebook back in 2012 to power their mobile applications. It is meant to provide a common interface between the client and the server for data fetching and manipulations. GraphQL was open sourced by Facebook in 2015.” — GraphQL vs REST

The single endpoint is a critical distinction. To get distributed, consolidated, or nested data a REST endpoint could, of course, invoke an integration service on the backend, or use techniques such as API Chaining. in GraphQL the “integration” exists semantically on the client query. The declarative query just describes the result and the server provides it. And since the shape determines the data desired at the single endpoint, the over/under fetching of REST is avoided. The query is ‘resolved’ on the server which may invoke of existing multiple SQL, REST services, MQ, or other services. This affords natural growth of the API. In contrast a REST API grows via more endpoints and/or addition of query parameters (which could morph to disguised RPC).

Image on www.graphql.com

“GraphQL was our opportunity to rethink mobile app data-fetching from the perspective of product designers and developers. It moved the focus of development to the client apps, where designers and developers spend their time and attention.” — https://code.facebook.com/posts/1691455094417024/graphql-a-data-query-language/

In “How GraphQL Replaces Redux” after the team started to use GraphQL they had three ‘Startling Realizations’:

  1. Most of our state management code was concerned with merging and mutating data from discrete REST resources into the right shape for our UI (reducers, selectors, actions etc.).
  2. A lot of our most complex state management was trying to manage the asynchronous nature of getting all that data in the right order for a specific route (sagas, middleware, thunks etc.).
  3. Practically everything else, UI state, worked great with plain old React state.

And, then they deleted a lot of code. A similar experience shown here: “Reducing our Redux code with React Apollo”, https://dev-blog.apollodata.com/reducing-our-redux-code-with-react-apollo-5091b9de9c2a

Summary

  • GraphQL is not a replacement, both can be used; even “GraphQL over REST”
  • GraphQL allows less chatty communication
  • GraphQL is not perfect
  • For simple API GraphQL will only add complexity.
  • Has been proven in many production applications and sites.

Critiques of GraphQL
New technology hype hardly ever gives the true efforts and failed approaches that it took to make something work. GraphQL is no exception.

1. “GraphQL Seems Like We Do Not Want to Do the Hard Work of API Design”, https://dzone.com/articles/graphql-seems-like-we-do-not-want-to-do-the-hard-w
2. “Just Because Github Has a GraphQL API Doesn’t Mean You Should Too”, https://www.programmableweb.com/news/just-because-github-has-graphql-api-doesnt-mean-you-should-too/analysis/2016/09/21
a. API chaining is fix for chatty REST?
3. “Are there any disadvantages to GraphQL?”, https://stackoverflow.com/questions/40689858/are-there-any-disadvantages-to-graphql?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
4. “Semantics and complexity of GraphQL”, https://blog.acolyer.org/2018/05/21/semantics-and-complexity-of-graphql/ Provides an algorithm for determining response size of a potential query.

Learning GraphQL
1. Introduction to GraphQL: http://graphql.org/learn/
2. “A Front End Developer’s Guide to GraphQL”, https://css-tricks.com/front-end-developers-guide-graphql/
3. “The Fullstack Tutorial for GraphQL”, https://www.howtographql.com/
4. Graphql.com tutorials: https://www.graphql.com/tutorials/
5. “Five Common Problems in GraphQL Apps (And How to Fix Them)”, https://medium.freecodecamp.org/five-common-problems-in-graphql-apps-and-how-to-fix-them-ac74d37a293c
6. “GraphQL Deep Dive: The Cost of Flexibility”, https://edgecoders.com/graphql-deep-dive-the-cost-of-flexibility-ee50f131a83d
7. GibHub explorer using GraphiQL: https://developer.github.com/v4/explorer/
8. “Thinking in Graphs”,
9. “Dan Schafer, GraphQL at Facebook”, https://www.youtube.com/watch?v=Ey3BtXDO2UM

Implementations
1. Prisma: https://twitter.com/GraphQL
2. Apollo client: https://www.apollographql.com/client/
3. Relay: https://facebook.github.io/relay/
4. GraphQL.js: http://graphql.org/graphql-js/
5. Apollo Server: https://www.apollographql.com/server
6.

YouTube Videos
*TODO*

Further Reading
1. “Is GraphQL Moving Toward Ubiquity?”, https://nordicapis.com/is-graphql-moving-toward-ubiquity/
2. “Top 5 Reasons to Use GraphQL”, https://blog.graph.cool/top-5-reasons-to-use-graphql-b60cfa683511
3. “GraphQL over REST at Reactathon 2018”, https://www.slideshare.net/sashko1/graphql-over-rest-at-reactathon-2018
4. “What are advantages and disadvantages of GraphQL, SOAP and REST?”, https://www.quora.com/What-are-advantages-and-disadvantages-of-GraphQL-SOAP-and-REST
5. “…And GraphQL for all?”, https://apihandyman.io/and-graphql-for-all-a-few-things-to-think-about-before-blindly-dumping-rest-for-graphql/
6. “REST versus GraphQL”, https://blog.pusher.com/rest-versus-graphql/
7. “Simplify relational logic with GraphQL”, https://scaphold.io/community/blog/querying-relational-data-with-graphql/
8. Who is using GraphQL? http://graphql.org/users/
9. “GraphQL is SQL for Knowledge, not Data”, https://react-etc.net/entry/graphql-is-sql-for-knowledge-not-data
10. “GraphQL vs REST comparison – Is This Solution from Facebook Worth Taking the Risk?”, https://www.netguru.co/blog/grapghql-vs-rest
11. “Sharing data in a Microservices Architecture using GraphQL”, https://labs.getninjas.com.br/sharing-data-in-a-microservices-architecture-using-graphql-97db59357602
12. StarBucks use of GraphQL: https://twitter.com/davidbrunelle/status/961103308957147136
13. “Richardson Maturity Model”, https://martinfowler.com/articles/richardsonMaturityModel.html
14. “2 Fast 2 Furious: migrating Medium’s codebase without slowing down”; https://medium.engineering/2-fast-2-furious-migrating-mediums-codebase-without-slowing-down-84b1e33d81f4
15. “Cartoon Guide to Facebook’s relay”, https://code-cartoons.com/a-cartoon-intro-to-facebook-s-relay-part-1-3ec1a127bca5
16. “Thinking in GraphQL”, https://facebook.github.io/relay/docs/en/thinking-in-graphql.html
17. The GitHub GraphQL API: https://githubengineering.com/the-github-graphql-api/
18. You don’t need a fancy framework to use GraphQL with React (All you need is a good naming strategy) : https://edgecoders.com/you-dont-need-a-fancy-framework-to-use-graphql-with-react-b47b436626fb#.hu577b1ln
19. “How GraphQL Replaces Redux”, https://hackernoon.com/how-graphql-replaces-redux-3fff8289221d
20. “Reducing our Redux code with React Apollo”, https://dev-blog.apollodata.com/reducing-our-redux-code-with-react-apollo-5091b9de9c2a
21. The GitHub GraphQL API: https://githubengineering.com/the-github-graphql-api/
22. https://twitter.com/GraphQL
23. “The Evolution of The New York Times Tech Stack”, https://stackshare.io/posts/evolution-of-new-york-times-tech-stack
24. “Designing a A True REST State Machine”, https://nordicapis.com/designing-a-true-rest-state-machine/
25. https://twitter.com/GraphQL
26. “The Evolution of The New York Times Tech Stack”, https://stackshare.io/posts/evolution-of-new-york-times-tech-stack
27. “Here’s my follow-up to REST is the new SOAP: let’s talk about the Original REST”, https://medium.freecodecamp.org/follow-up-to-rest-is-the-new-soap-the-origins-of-rest-21c59d243438
28. “GraphQL vs REST: Overview”, https://philsturgeon.uk/api/2017/01/24/graphql-vs-rest-overview/
29. “Reflections on REST — FSE’17 Keynote”, https://www.youtube.com/watch?v=6oFAmQUM8ws&feature=youtu.be
30. “A Journey Through React Apollo – Peggy Rayzis aka @peggyrayzis at @ReactEurope 2018”, https://www.youtube.com/watch?list=PLCC436JpVnK3xH_ArpIjdkYDGwWNkVa73&time_continue=1&v=fCXYA3lZTbo
31. “Dan Schafer – GraphQL: Client-Driven Development”, https://www.youtube.com/watch?v=vQkGO5q52uE
32. “A Real-World GraphQL Application in Production”, https://www.youtube.com/watch?v=6bRFElKG0a8
33. “What it takes to run the world’s largest GraphQL server”, https://www.youtube.com/watch?v=nOYyix0NpvY
34. “From REST To GraphQL”, https://www.youtube.com/watch?v=ntBU5UXGbM8&t=690s

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Declarative Object Graphs: programs as queries

If there are no physical constraints could a program look up it’s solution by firing off a set of queries into a solution space?

While taking a shower this morning I was pondering about some things I read recently.

First some history. Programs were created to run in constrained resources. As such, a program is just collection of small bits of data transforms. Yes, more complicated than this, don’t interrupt.

What if there were no constraints? In future we will have more memory and network speed will be much better.

With no constraints we could map inputs into outputs directly, no computations. A program becomes a query into possible solution spaces. A functionless approach.

This is already the case with database technology. For example, relational databases use a Relational Algebraic basis for declarative programming. In some data centers a whole database of terabyte size fit in memory.

What is required is a mathematical basis for non-constraint programming. Here is my intuitive notion. A program is a dot in a vast tensor state space. When a user clicks on a button, that dot moves to a different subspace and all it’s queries are evaluated in with new basis vectors. When the dots are related to other dots, we have a system.

Since a program is just a bunch of queries from an “instance” to a ‘space’, it should be possible to use big data techniques (“AI”) to generate that space. This would make it possible to automatically generate applications based on a transition graph of how objects move through space.

Well, I have no time or expertise to make the above work. It was probably some chemical in the soap. Doesn’t seem possible to create such a space.

Edits
20180207T0540: Added stuff about automatic program generation.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Hanging basket with a bunch of solar powered lights

Solar powered landscape lighting is an inexpensive way to add a simple accent to any ground area.

I bought a box of 12 lights and the cost was about 80 cents per unit. I didn’t have time to put them where I wanted. I just unpacked them and put them in a hanging basket on the deck.

Just a bunch of solar lights
Just a bunch of solar lights

Wow, I accidentally created a beautiful effect. In a very dark night JBOSL (Just of Bunch of Solar Lights) gives a soft eerie accent light.

They don’t even have to be in direct sunlight during the day. You don’t want bright lights, just a glow. Of course, if they don’t charge enough the glow won’t last long.

There are actual hanging solar powered light products out there, but for 80 cents a light and a cheap basket, you could add accents all over the place. Not too much, or you’ll be visited and have a close encounter of the wrong kind.

Caveats
Solar lights are not very reliable. First they are cheaply made and I think quality control is weak. If you search web, you’ll find constant complaints that a percentage of solar lights received don’t even work.
What I find the most painful is that the batteries used in these things are “non-standard”; so hard to replace. Yup, rechargeable batteries wear out.

Related links

  • 5 Main Reasons Why Your Solar Lights Are Not Performing as Well as New.
  • Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    DystopianFuture = f( Robots, BigData, AI, IoT )

    I was standing at the window watching the recycle trunk pick up our container. The trunk was driven by one person who just operated a giant robotic gripper that extended from the truck to hold the container and empty it. Took about five seconds. Up and down our street the truck zoomed. All done. Amazing. Before, there were a few (strong!) people who did this. It was one driver and two other workers. Where are they now?

    Currently, only recycle bins in my city are standardized. So, a truck with a single manipulator is usable. In future, its inevitable that the rest of the garbage will be disposed in standard containers, and more people will do less of this manual labor. Nothing new, the march of automation and more powerful computers. For decades there have been those warning of dire consequences, yet we are still here and have more of more stuff. So what is the problem?

    There is a confluence of technological breakthroughs that are bound to happen and all these are being quickened by the internet. Internet speed change is occurring. Robotics breakthroughs and applications are only waiting for a new form of power storage and control.

    Currently, all advanced technologies are limited. Drones have limited range and robots are mostly lumbering beasts that fall over doing the easiest tasks that humans can do while sleep walking. Yet, this won’t always be the case. In fact, the substitution of humans with autonomous intelligent systems is not even the only way this Change will happen. It could be that the path will just be human augmentation.

    People can be augmented to use more AI and robotics technology. This will increase productivity in many fields. Thus, we’ll need less people. Even now we see the effect of more information in knowledge industries. Years ago a software developer had to remember many things, but now with the web the admin-techno-config-trivia is available at a mouse click. The bell curve of who can program has shifted. Since knowledge is available, silos of expertise are reduced. Now one person can do many different things. Again we’ll need less people.

    If we need less people, how will the economic system function? Does it become a giant welfare state where only a few do “meaningful” work and the others have a form of guaranteed sustenance? Or will it become a dystopian nightmare of a stratified classes, the majority are the new ‘untouchables’, the middle tier occupied by the technocratic knowledge workers, and all presided over by the upper 1% ruling class?

    Links
    obots’ Next Big Job: Trash Pickup

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    React component with Media queries for Responsive

    This is a very simple approach using matchMedia support. We store media queries and a listener function as an object in an array. When the media query returns true the listener function will set the relevant component state true.

    In our actual project, this approach is used with Redux. The queries instead of setting component state, update the Store. Containers connected to this store send the breakPoint, orientation, and other props to each child component.

    Just an experiment. If pursued further, would probably make it a HOC (Higher Order Component).

    Gist available here.

    Demo
    If packed and loaded into a web page as we resize the browser we should see:

    Change size of browser to test
    
    state: { 
        "orientation": "landscape", 
        "breakPoint": "LG" 
    }
    

    Why?
    While CSS media queries are very useful, sometimes being able to respond ‘behaviorally’ to media events is very powerful.

    Tested on
    Chrome, FireFox, IE, and Edge. I guess they all have JavaScript MediaQuery support.

    How it works

    In the life cycle method ‘componentDidMount’ we add a query to an array in the component:

    this.mediaQueryFactory(obj, '(orientation: portrait)', this.onPortrait);
    
    this.mediaQueryLists.push({
    	mql: obj.mql,
    	listener: this.onPortrait
    });
    

    the onPortrait is the listener function:

    onPortrait : function(flag){
      if(flag){
    	this.setState({orientation: 'portrait'});
      }
    },
    

    Full Code

    Other MediaQuery React implementations

    Addendum

    {
      "name": "ReactMediaQuery",
      "version": "1.0.0",
      "description": "Attempt at using Media Queries in React",
      "main": "./index.js",
      "scripts": {
        "test": "echo \"Error: no test specified\" && exit 1",
        "start": "webpack index.js dist/react-media-query-bundle.js"
      },
      "keywords": [
        "react"
      ],
      "author": "JosefBetancourt",
      "license": "SEE LICENSE IN COPYWRITE.TXT",
      "dependencies": {
        "babel-polyfill": "^6.9.1",
        "classnames": "^2.2.5",
        "react": "^15.3.0",
        "react-dom": "^15.3.0"
      },
      "devDependencies": {
        "babel-core": "^6.10.4",
        "babel-loader": "^6.2.4",
        "babel-plugin-add-module-exports": "^0.2.1",
        "babel-preset-es2015": "^6.9.0",
        "babel-preset-es2016": "^6.11.3",
        "babel-preset-react": "^6.3.13",
        "babel-preset-stage-2": "^6.11.0",
        "eslint": "^3.2.2",
        "webpack": "^1.13.1"
      }
    }
    

    package.json

    var path = require('path');
    const webpack = require('webpack');
    
    module.exports = {
      /* entry point */
      entry: './index.js',
      /* output folder */
      output: {
        filename: 'dist/react-media-query-bundle.js'
      },
      devtool: 'source-map',
      module: {
        loaders: [
          {
            test: /\.js$/,
            loader: 'babel-loader',
            exclude: /node_modules/,
            query: {
              presets: ['es2015', 'react', 'stage-2']
            }
          },
        ]
      },
      eslint: {
        configFile: path.join(__dirname, 'eslint.js'),
        useEslintrc: false
      },  
      plugins: [
      ],
      resolve: {
        extensions: ['', '.js', '.json'] 
      },
      resolveLoader: {
          modulesDirectories: [
              'node_modules'
          ]
      }
    };
    

    webpack.config.js

    <!DOCTYPE html>
    <html>
      <head>
    	<meta charset="utf-8">
    	<meta name="viewport" content="width=device-width, initial-scale=1.0">  
      	<title>React Media Queries</title>
      </head>
      <body class="body">
        <div id="root" class="">Loading page, please wait ....</div>
    	<script src="dist/react-media-query-bundle.js"></script>
      </body>
    </html>
    

    index.html

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    Version control console using ReactJS

    Created a web app to generate a report from the version control repository, Apache Subversion™. Similar approach is possible targeting a different repository, like Git.

    TL;DR
    Someone said a process we follow could not be automated. I took that as a challenge and created a proof of concept (POC) tool.

    The final GUI using ReactJS is relatively complex: five data tables that hide/show/expand/collapse. Searches on those tables, sorting, navigation links, help page, Ajax requests to access Subversion repo data, export to CSV, report generation, and client side error handling. It started as just a GUI to a report, but since it was easy, added more features: Zawinski’s law.

    To top it off, the app had to automatically invoke the default workflow or no one would use it.

    Result?
    1. It is a complex disaster that works. And works surprisingly fast. Using ReactJS and Flux made it into a fast elegant (?) disaster that works … kind of.
    2. The app served as an example of a SPA in the our dev group. But, mostly to try out the ReactiveJS approach.
    3. My gut feel is that there are some fundamental problems in the client side MV* approach which leads to control flow spaghetti (a future blog post).

    Since the time I wrote that app I have noticed a mild push back on React that point out the hidden complexities. There are now new ideas and frameworks, like Redux or Cycle.js. Most recently, to tackle the action-coordination, there is much digital ink written on Redux Sagas, for example: “Managing Side Effects In React + Redux Using Sagas“.

    Note, though there are critiques of the ReactJS approach or implementation, this does not imply that React was not a real breakthrough in front end development.

    Report generation
    Creating simple reports from a version control repository can be accomplished with command line tools or querying XML output from SVN log commands. In this case generating the criteria to generate the report was the hard part. Details are not relevant here: this web app would remove a lot of manual bookkeeping tasks that our group currently must follow due to a lot of branch merging and use of reports for error tracking, verification, and traceability. Yup, long ago legacy Standard Operating Procedures (SOP) of an IT shop.

    Architecture
    Server
    A simple Java web app was created and deployed to a Tomcat server. A Java Servlet was used at the server to receive and send JSON data to the browser based client. This server communicates with the version control repository server.

    Client
    The browser is the client container with ReactJS as the View layer and Flux (implemented in the McFly library) as the framework client implementation. Dojo was used as the JavaScript tools library. Dojo supplied the Promise and other cross-browser capabilities. Why Dojo? That is already in use here. If we were using JQuery, that is what I would use.

    Local application service
    Performance note: Since the repo query and processing occurs at the server, multiple developers accessing the service would have a performance impact. A future effort is to deploy this as an runnable Jar application (Spring Boot?) that starts an embedded app server, like Tomcat or Jetty, at the developer’s workstation. The browser would still be used as the client.

    Repository Query
    Some options to generate SVN reports:

    1. Use a high level library to access SVN information.
    2. Export SVN info to a database, SQL or NoSQL.
    3. Use an OS or commercial SVN report generator.
    4. Use command line XML output option to create a navigational document object model (DOM)
    5. Use SVN command line to capture log output, and apply a pipeline of Linux utilities.

    This was a ‘skunkworks’ project to determine if some automation of a manual process could be done and most importantly, if doable, would the resulting tool be used? The first option, seemed easiest, and was chosen. The repo was accessed with the SvnKit Java library. (For Java access to a Git repo, JGit is available).

    The process approach was to generate and traverse a Log collection. A simple rule engine was executed (just a series of nested conditionals) to determine what to add from the associated Revision objects.

    This seemed like a workable idea until a requirement was requested after the POC was completed: instead of listing a particular source file once per report, show multiple times per each developer who made a commit to it. An easy change if this were implemented as an SVN log query sent to a pipe of scripts. However, with the design this required going into the nuts and bolts of the “rule engine” to add support for filtering, and further changes to the model.

    Yup, a POC solution can be a big ball of mud, and unfortunately can be around a long time. Incidentally, this happened with Jenkins CI; where I …

    Very recently a flaw in the design will force a revisit of the algorithm again. Instead of making the ‘rule engine’ more powerful, an alternative approach is to start from a Diff collection. The diff result would be used to navigate the Log collection. A similar approach is shown here: http://www.wandisco.com/svnforum/forum/opensource-subversion-forums/general-setup-and-troubleshooting/6238-svn-log-without-mergeinfo-changes?p=36934#post36934

    But, already a problem was found with diff output. There is no command line or Java library support for pruning of deleted folders. For example, if a/b/c is a hierarchy, and you delete b, c is also deleted. Now if you generate a diff, the output would contain delete entries for: a/b and a/b/c. What was needed was just a/b. Simple, you say. Sure, but this information is a OOP object graph, so can be complicated.

    I solved it: a diff list of 1800 folders was reduced to just 8 folders! I’m surprised a solution or something similar was not found in a web search. I wrote about this in “Can conditional state be used in RxJava Observable streams?

    Perhaps revisit the alternative approaches, like export to database? Not sure if this really would have simplified things, but instead just change where the complexity was located. Is the complexity of a software solution a constant?

    Other systems take this export approach. One system I saw years ago, exports the version control history (it was CVS) into an external SQL database and then used queries to provide required views.

    Client Single-Page Application
    What to use as the browser client technology? From past experience, I did not want go down the path of using event handlers all over the place and complex imperative DOM updates.

    Anyway, React seemed interesting and had a shorter learning curve. I looked at Angular, but it seemed to be the epitome of embedding the developer into the product (future blog post on the application development self-deception).

    A few ReactJS components were created:

    • BranchSelect
    • CommentLines
    • ControlPanel
    • DiffTable
    • ErrorPanel
    • ExclusionRow
    • ExclusionTable
    • FilesRow
    • FilesTable
    • ManifestRow
    • ManifestTable
    • ProgramInfo
    • ProjectPanel
    • RevisionRow
    • RevisionTable
    • ViewController

    Lessons Learned
    This project progressed very quickly. React seemed very easy. But, that was only temporary. Until you understand a library or a paradigm, start with a smaller application. Really understand it. Of course, these too can fool you. For example, when this app first loads, I had to invoke the most likely use-case. There was a endless challenge of chicken/egg model flow disasters. Solved it, but can’t understand how I did it. Somehow I tamed the React flow callbacks. Or this is just a lull and will blow up in as a result of an unforeseen user interaction.

    All the new cutting edge JavaScript front-end frameworks are very very complex for us average developers. Just the tooling is daunting if your coming from a vanilla JavaScript shop. Node dis! See this: ‘I’m a web developer and I’ve been stuck with the simplest app for the last 10 days

    Next SPA project?
    My next app will probably use Redux as the Flux framework. Or may leave Reactjs and go directly with Cycle.js which is looking very good in terms of ‘principles’ or conceptual flow and is truly Reactive, based on a ReactiveX library: RxJS.

    Links

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    RxJava – Chaining Observables

    In ReactiveX one can chain Observables using the flatMap operator, similar to chaining Promises using “then“.

    Example 1
    In listing 1, three reactive streams are created.
    – counterStream creates a stream of integers 0, 1.
    – helloStream creates a stream three strings, “Hello”, “World”, “!\n”
    – resultStream chains helloStream to counterStream.

    Listing 1

    public class ChainObservable {
        /**
         * @param args  
         */
        public static void main(String[] args) {
            Observable<Integer> counterStream = 
               Observable.range(0, 2);
            
            Observable<String> helloStream = 
              Observable.create(o -> {
                o.onNext("Hello");
                o.onNext("World");
                o.onNext("!\n");
                o.onCompleted();
            });
            
            Observable<Object> resultStream = 
              counterStream
                .map(x -> {
                    System.out.println("# " +x);
                    return x;
                })
                .flatMap( counter -> 
                   helloStream
                );
            
            resultStream.subscribe(
                v -> System.out.println(
                      "Received: " + v
                ),
                e -> System.out.println(
                      "Error: " + e
                ),
                () -> System.out.println(
                       "Completed"
                )
            ).unsubscribe();
            
        }
    }
    

    The output of listing 1 is:

    # 0
    Received: Hello
    Received: World
    Received: !
    
    # 1
    Received: Hello
    Received: World
    Received: !
    
    Completed
    

    Note that the “chaining” or ‘thening’ used here is not quite what chaining was meant for. The flatMap operator use in listing 1 passes the current counter, but the chained Observable does not use it, the Observable just repeats its onNext(…) invocations. The flatMap: “transform the items emitted by an Observable into Observables, then flatten the emissions from those into a single Observable”

    The strange thing is the “Completed” output. The code does this because of line 14 in the source (does it?). The helloStream invokes onCompleted, but the completed in the subscriber is not triggered until the final counterStream event. Or, I’m looking at this incorrectly?

    Example 2
    In example 1 above, the function that operates on each item passed by the source Observable is not used. I’m wondering if it could be used in a “chained” in Observable, as in listing 2 below. Does this make sense? Doesn’t this then have a runtime penalty since the Observable is not created before it is used?

    Listing 2 version where Observable uses flatMap data

    import static java.lang.System.out;
    import rx.Observable;
    
    /**
     * 
     * @author jbetancourt
     *
     */
    public class ChainObservable {
        /**
         * @param args  
         */
        public static void main(String[] args) {
            Observable.range(0, 2)      
            .map(x -> {
                out.println("# " +x);
                return x;
            })
            .flatMap( count -> 
              Observable.create(o -> {
                o.onNext(count + " Hello");
                o.onNext(count + " World");
                o.onNext(count + " !\n");
                o.onCompleted();
              })
            )
            .subscribe(
                v -> out.println("Received: " + v),
                e -> out.println("Error: " + e),
                () -> out.println("Completed in subscribed")
            ).unsubscribe();
        }
        
    }
    

    Output of this version is:

    # 0
    Received: 0 Hello
    Received: 0 World
    Received: 0 !
    
    # 1
    Received: 1 Hello
    Received: 1 World
    Received: 1 !
    
    Completed in subscribed
    

    Links

  • RxJava
  • RxJava: chaining observables
  • Grokking RxJava, Part 2: Operator, Operator
  • Transformation of sequences
  • Don’t break the chain: use RxJava’s compose() operator
  • Implementing Your Own Operators
  • Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    Sonos music using external HD on Raspberry Pi

    You can attach an external hard drive to a Raspberry Pi and then share music over Sonos. This works very well. Even though my hard drive is connected to the rPI via USB 2.0, the music streams fine, no stutters.

    Right now I’m playing Jeff Buckley’s ‘Sketches for My Sweetheart The Drunk’ all over the house. “Vancouver” track is so awesome!

    Technically this kind of storage sharing falls under the term Network Attached Storage (NAS). But, that seems like an overblown term for just sharing one disk. There are a lot of features on a full-blown NAS.

    How does the RaspberryPI share the storage? By running a server called Samba. This is a set of open-source programs that run in Unix/Linux to provide file and print services compatible with Windows-based clients.

    Spin down?
    Currently I’m looking into how to enable spin down of the hard drive when idle. Necessary? Supposed to make HD last longer. I just want to reduce power usage. The whole point of a Raspberry Pi in this scenario.

    Maybe this page, “Spin Down and Manage Hard Drive Power on Raspberry Pi”, will help.

    Update:
    Feb 1, 2015
    hdparm and sd-idle did not work. trying SDPARM.
    sudo blkid
    shows:
    /dev/sda1: LABEL=”USB B” UUID=”B23A8B373A8AF81D” TYPE=”ntfs” PARTUUID=”b75ac8d0-01″

    sudo sdparm –flexible –command=stop /dev/sda1
    shows:
    /dev/sda1: ST325082 3AS

    Feb 3, 2015
    The spin down using sdparm is working fine.

    Hardware Used

    • Raspberry Pi 2 (Canakit Ulitimate Starter Kit with WIFI)
    • Old USB hub from Staples
    • 3.5 inch 1 GB Hard drive
    • HexStar-3 3.5in External Hard drive enclosure

    Technical details
    I had a lot of grief getting it to work. Haven’t touched a Linux system in while.

    Some articles of the many articles I found information on how to do this are in the links section below. Note that there isn’t one single approach to do this. And, it also depends on what OS your running on Raspberry PI. I’m running Raspbian which I installed via NOOBS; all included in the kit I purchased.

    Links
    Other articles about this approach:

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    Use AOP with RxJS to make functions Observable

    There are no Observable create functions that will observe function invocations on an Object. As a proof of concept I used a JavaScript AOP library, Meld, to accomplish this “function to stream” capability.

    Alternatives
    This sounds like a job for a mixin, proxy, Object.observe, inheritance, or decorator pattern. There are hundreds of mixin libraries, the new JavaScript Proxy is not readily available, Object.observe was cancelled, etc.
    Edit: Via a Dan Abramov tweet (what a busy guy!), I found out about MobX.

    Aspect-Oriented Programming (AOP)
    AOP is paradigm that allows the separation of cross-cutting concerns via applying “advices” to objects. There are many advice types. The Meld library supports several, for example, ‘before’, ‘around’, ‘on’, ‘after’, ‘afterReturning’, and ‘afterThowing’.

    We’ll just use two here: “after” to observe a function invocation, and “around” to observe a property change.

    Observe of function
    In this example object:

    var myObject = {
       myProperty:'15',		
    
       doSideEffect: function(x){this.myProperty = x;},		
    
       doSomething: function(a, b) {return a + b;}
    };
    
    

    We want to subscribe to the ‘doSomething’ function in this object. That is, when that function is invoked, the resulting return value should be sent to an Observable stream.

    The stream:

     
    var subject = new Rx.Subject();
        
    var subjectAsObservable = subject.map(function(x){
           console.log("in map x:" + x);
           return x;
    });
        
    var subscription = subjectAsObservable.subscribe(
           function(x){ // onNext
               console.log("Next in subject: " + x.toString());
           }	
    );
    

    Now to apply an After advice to that function “joinpoint” we do:

     
    var remover = 
      doAfter(myObject, 'doSomething', subject, 
        function(result) {
            console.log("after function .... ");
        }
    );
    
    myObject.doSomething(1, 2); 
    remover.remove();
    

    The output of running this is:

     
    in map x:3
    main.js:3
    after function .... 
    

    Note that “When adding advice to object methods, meld returns an object with a remove() method that can be used to remove the advice that was just added.”

    “doAfter” function is a simple function wrapper of the ‘after‘ Meld function:

     
    /**
     * @param anObject the object to advise
     * @param method the method selector
     * @param subject the subject that observes the function
     * @param afterFunction function to invoke after the advised function returns
     * @return object with remove() method
    function doAfter(anObject, method, subject, afterFunction){
        return meld.after(anObject, method, function(result){
        	subject.onNext(result);
        	afterFunction(result);
        });
    }
    

    Observe of property change
    Some AOP libraries support get/set advices on property changes. AspectJ in Java world supports this. Meld does not, but this can be implemented using an Around advice. This type of advice wraps a function call.

    With an Around advice we can record a properties’ value before and after a function invocation. If there is a change, we generate a new event for the Observable stream.

     
    remover = meld.around(myObject, getMethodNames, function(methodCall){
        var originalValue = myObject.myProperty;
        
        var result = methodCall.proceed();
        if(originalValue != myObject.myProperty){
            subject.onNext("prop changed: " + originalValue + " -> " + myObject.myProperty);
        }
        
        return result;
    });
    
    myObject.doSideEffect('25');
    
    remover.remove();
    

    The above uses a utillity function:

    /** Utility function to get all functions in object */
    function getMethodNames(obj){
       return Object.getOwnPropertyNames(obj)
            .filter(function(name){
    	    return typeof obj[name] === 'function';
            });		
    }
    

    When any method in object is invoked that changes the property, we get:

    in map x:prop changed: 15 -> 25
    Next in subject: prop changed: 15 -> 25
    

    Of course, there is a performance cost to this approach of monitoring property changes. This costs has not been quantified or explored.

    Summary
    Using AOP an object’s functions can be made into observers. An “after” advice can signal a result event, and using an “around” advice, it is possible to generate events on an Object’s properties. Useful? Unknown. Already possible in ReactiveX ecosystem, not sure.

    Possible operators for the above could be:
    fromFunction(object, methodSelector)
    fromProperty(object, propertySelector)
    fromProperty(object, methodSelector, propertySelector).

    TODO
    Look at the Rx.Observable.fromCallback(func, [context], [selector]) operator. Could it be used to go from function to Observable stream without using AOP, as presented here?

    Software used
    RxJS: version 4.0.7, rx.all.compat.js
    Cujojs Meld: version 1.3.1,

    Links

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    RxJS process loop with toggled operations and canceling

    How to create a stream on a button that toggles between two operations, and another button that can cancel the same stream? Observable operators are numerous and very complex, so coming up with a solution, or composing a stream, to any problem can be very difficult. At least, that is what I found while learning all this reactive stuff.

    Use case
    UI:
    Two buttons: one labeled “Start”, and another labeled “Exit”.

    Behavior:
    First click on ‘Start’ button, changes its label to “Toggle” and starts the operational loop that invokes operation1.

    Further click on button, now labeled “Toggle”, will change the operation invoked alternatively to operation2 or operation1.

    A click on the ‘Exit’ button will stop the loop stream.

    Here is the RxJS based solution running in JS Bin playground: (Ability to re-run is not implemented. ‘Exit’ means you closed the app.)
    RxJS toggable and cancellable process loop on jsbin.com

    Sure, this would have been much easier with just regular JavaScript. Just create a timer and a listener on each button, etc. Observable streams don’t really add any value for this simple scenario. But, this is just a learning exercise.

    Then again, using streams may offer more opportunities for handling more complex scenarios with timer loops.

    Similar
    This appears to be just like the very basic traditional state machine exercise of modeling a stopwatch. There, the toggle button starts/stops the timer, and the Exit is the cancel button. Do a web image search for “statechart stopwatch”.

    Jump to source code

    Solution
    This seems easy now, but it took a lot to come up with a solution. The approach selected, using ‘scan’ was suggested in an article and a StackOverflow question. ‘Scan’ can be used to transfer statefulness.

    For the loop we can create streams using timer or interval.

    The alternation of operation is where the complexity comes in. “case” operator is very powerful since it can choose an Observable based on a key into a sources map. Would be useful if the operations, i.e., functions invoked in the loop, are themselves streams or Subjects.

    If there is only one operation to execute in the loop and the toggle button is behaviorally a run/pause, then there is the “pausable” operator.

    One thing I did not implement is a way to restart the process after it was exited. Do the whole thing as a programmatic setup on first start click, or figure out how to resubscribe to the observables? Oh well.

    Software used
    RxJS: version 4.0.7, rx.all.compat.js
    JQuery 2.2.0: jquery-2.2.0.js

    Links


    Source
    Two versions shown below, ES6 using arrow function expressions, and one using plain functions. I’m sure this is a naive newbie solution to this, so be very skeptical of the RxJS usage here. Note: I named the streams with a “$” prefix. a kind of reverse Hungarian Notation. This is a naming convention used by the Cycle.js project.

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    The celebrated man in the street

    %d bloggers like this: