We’re releasing our first Technology Radar. A Technology Radar, pioneered by ThoughtWorks, is a list of the techniques, tools, platforms, languages, and frameworks divided into recommendation levels: Adopt, Trial, Assess and Hold. It’s mostly a list of things we find useful and interesting.
We’ve built our own Technology Radar for several reasons. We think it will be useful internally to help formalize our opinions on various technologies and guide our decisions on technology adoption. It’s also a useful metaphor for communication. Customers and fans of element 84 may find it useful as well. This is a trial for us. It may be our last radar or the first of many. We’ll see what the benefit is and how motivated we are to continue producing it in the future.
Our recommendations are based on our experience (or lack thereof) with a technology. We may put a tool that’s been around for a while in Assess and not Adopt if we’ve done some testing with it but haven’t made the decision yet whether it’s ready for production. If you don’t agree with our recommendation we may just have different goals or lack similar experiences to come to the same conclusions. n The four recommendations levels mean the following:
- Adopt – We recommend adopting this technology in production where it is appropriate.
- Trial – This tool is worth prototyping and testing internally. It may be production ready in some cases.
- Assess – This is a new technology (or new to us) that may not be production ready. It is worth reading about and understanding.
- Hold – We recommend avoiding future use of this technology in new applications.
Techniques
Adopt
Functional Programming
Functional programming has had a swell in popularity in recent years. We recommend using functional approaches when possible. Functional programming among other things avoids mutable state and tends to make it easier to write simple, concurrent code.
Dedicated Error Tracking
Tools such as Airbrake, Errbit, and Raygun provide dedicated tracking of software faults. These tools are useful for tracking server side errors but essential for identifying and fixing problems in mobile and web apps.
Testing at the appropriate level
This is a technique previously identified on the Thoughtworks Radar in March 2012. It’s important not to overdo integration tests which are very expensive to maintain. It’s also easy to try to jump through too many hoops to unit test functionality that’s tested best at the integration level. We repeat the advice from Thoughtworks because we feel strongly about avoiding this trap.
Testing Distributed System Failure Cases
Distributed systems are complex and easy to get wrong. The testing Kyle Kingsbury has documented at aphyr.com show that vendors sometimes overpromise on their abilities to handle distributed system faults. Inspired by Kyle, we’ve adopted similar approaches during our testing of various systems. We recommend identifying areas where OSS or COTS software is critical to your business and testing the failure cases that matter to you.
Property Based Testing
Traditional unit testing with example based tests test individual examples created by the test developer. Property based tests defines the shape of input for a function and properties that must hold true for all input. When a property based test is run random values are generated that conform to the defined input and the properties are checked to hold true. Property based tests will test many more values than example based tests. n Implementations of property based testing are available in many languages. We’ve had success in Clojure using test.check (previously called Simple Check) created by Reid Draper.
UI/front-end documentation and samples
Web applications, mobile apps, and even desktop apps need to define a consistent visual design, components, and style. One technique we have found that works very well is to write UI documentation using the same visual styling of the application itself. The documentation describes and shows how each of the various components are used, how pages look and feel, and what the visual metaphors are and how they’re used. This saves a lot of time during development. Instead of a developer trying to guess at widgets and style questions they have a resource to use. It provides a communication channel between designers and developers. The documentation can grow as new styles, metaphors, and widgets are needed.
Daily Journaling
Many engineering disciplines emphasize note taking as an important skill. Notes are sometimes even legally required in some patentable research and development work. Computer Science and related IT fields have not placed the same emphasis on note taking. We recommend keeping a daily journal of things worked on, problems encountered and solutions, and ideas. This has many benefits for retaining information and avoiding repeating the same mistakes. Past notes and ideas have a way of compounding into bigger ideas. There are may applications like Day One that provide syncing and backup of notes across devices.
Trial
Visualization Driven Development
Visualizations have proven themselves incredibly useful for understanding data and solving problems. Visualization Driven Development is the practice of applying visualizations to everyday development. See Jason Gilman’s talk from Strange Loop 2013. This technique is promising but we recommend slow adoption while learning when to apply it, best practices, and what tools to use.
Assess
CommonJS or AMD in Rails projects
The Rails asset pipeline encourages bad practices such as passing information through globals. It also doesn’t follow accepted Javascript conventions for defining modules and dependencies. As a consequence, it is difficult to produce and maintain a large modular Javascript codebase in Rails. Ruby gems exist for using more standard approaches such as CommonJS or AMD. It is also possible to abandon the asset pipeline altogether and use build tools such as Grunt to produce Javascript.
Tools
Adopt
Serverspec
Serverspec allows you to write RSpec tests that verify that your servers are properly configured. This is very useful for verifying security settings and system configuration. It helps quickly verify that newly provisioned machines meet their requirements.
Enlive for server-side templating
Enlive is a selector based templating library for Clojure. It allows decoupling markup and the rendering logic. It’s a bit of a brain bending experience to learn, but once you “get it” it is extremely powerful.
Grunt
Grunt is a build system for node.js applications. It takes care of a lot of the typical web application build steps like compiling, minifying, versioning. It allows expressing tasks, order of tasks and dependencies between them. Alternatives build systems like Make will work but require more of the bootstrapping of basic tasks that Grunt provides out of the box or in one of the many plugins available.
Leaflet.js
Leaflet is a JavaScript Library for mobile-friendly maps. Leaflet is lightweight with good extensibility. We’ve found it very easy to tweak to make it work for our own needs. It’s simple to setup but works well for more complex mapping problems. Leaflet is built for modern browsers and best practices on the web. It’s our preferred web mapping solution instead of OpenLayers or Google Maps.
Gradle Build System
Gradle is a build system using a Groovy based DSL. Gradle has the flexibility of Ant and the convention over configuration style of Maven. It’s now the officially recommended build system for Android projects.
Trial
Packer
“Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.” We recommend it for use with tools like Chef and Puppet to automate the creation of machine images.
Genymotion (Emulator)
Genymotion is an Android Emulator. It has a wide range of AOSP images available and is extremely fast.
core.async
core.async is a library for Clojure and ClojureScript that provides CSP style channels similar to the language level feature in the Go programming language. It makes conveyance of messages between components of an application a first class citizen using channels. This avoids the “fragmented logic” or Callback Hell when using non-blocking APIs or event driven APIs. core.async is still marked as alpha. It is limited to in-process communication but it can be combined with an external message queue implementation for communication between processes.
Custom font icons sets
Icon fonts like FontAwesome have already proven in production systems and make dealing with a myriad of icons easier. Custom font sets let you only include the glyphs you need decreasing the bandwidth and processing necessary to display them on the page. Overall performance and maintainability needs to be addressed as some of the icon sets are generated from JSON files that need to be versioned and shared. Documentation is key as other designers and developers need to know where and how to use the font icon set.
Knockout.js
http://knockoutjs.com Lightweight, fast, highly-configurable, with the potential to be unobtrusive, this Javascript MV* framework provides significant utility with a shallow learning curve and without the tendency to take over a codebase like similar frameworks.
Assess
Docker
Docker is built on LinuX Containers (LXC) and allows creation of lightweight application containers. This is similar to the idea of virtual machines but is a lighter weight solution. Docker allows safe sharing of common resources between containers like the base operating system but provides more isolation than putting multiple applications on a single system. Restarting a container is much faster than rebooting a whole operating system. Hundreds of these containers can run on the same physical host or VM because they are so lightweight. n Source: http://stackoverflow.com/questions/16047306/how-is-docker-io-different-from-a-normal-virtual-machine
New WYSIWYG tools
These tools, such as Macaw, have matured greatly since products like Dreamweaver and FrontPage, often incorporating features for building responsive and HiDPI-ready designs. Markup and style output needs to be examined for code quality before moving to production. Integrating into a current design flow may be problematic. Questions like how to integrate software like Macaw into a larger process must be answered.
Spoon
Spoon is an Android testing tool that allows distributing tests among multiple devices concurrently.
Hold
Cucumber Testing
Cucumber is a behavior driven development tool that allows writing tests in plain english. Test implementation ties to the test descriptions through the use of regular expressions associated with blocks of code. We’ve dedicated many hours to extensive suites of cucumber tests. We encountered many issues with Cucumber test maintainability and difficulty debugging them. Developers also have a tendency to “Code in English” when using Cucumber. The concept is interesting but there’s a lot of overhead in writing tests this way and matching them individually with regular expressions. We haven’t found that the notion that customers or analysts will want to read the feature files to hold true. That might be the case in other companies. We recommend using a general testing library like Rspec, clojure.test, etc to write integration tests in most cases.
Platforms
Adopt
Elasticsearch
Elasticsearch is a distributed search engine built on top of Lucene. We’ve been using it in production for two years and have been happy with it’s performance and capabilities. There are many tools that have been built on top of it and tools for monitoring it.
Node.js
Node.js is a platform for execution of JavaScript on Google’s V8 JavaScript engine. Node.js has been improving and advancing over the years. It’s very fast and has a lot of third party packages. For web applications it allows a consistent language on the client and server side. It also has the potential for sharing code on the client and server side. There are many tools built on top of Node. Even if you’re not using it for building an application it should probably be in your local development tool chain.
Microservices
Microservices are a software architectural style that emphasizes the use of multiple small decoupled processes as opposed to a larger monolithic system. Martin Fowler has one of the best descriptions of microservices on his blog. We’ve found microservices to be a useful way to divide up a system by responsibility. The smaller services are usually easier to understand, debug and test. They also offer greater flexibility in our increasingly polyglot field. New languages and technologies can be tested on a single service without impacting the rest of the application. The cost of a wrong decision becomes much smaller within a microservice. n Care must be taken to divide up capabilities into cohesive services that have low coupling. The use of microservices will usually increase complexity at the system level in places like deployments and maintaining highly availability of services.
Trial
Immutable Data Storage
The idea of append-only or immutable data storage has been around for a long time. It has recently become more popular through the emergence of products like Datomic and architectural patterns like Event Sourcing. Immutable storage has many benefits. Data that doesn’t change is easy to keep in sync. This has benefits for caching, synchronization between mobile and server data, and using eventually consistent databases like Riak. Immutable data also maintains history which provides an audit trail of changes. n Immutability should be considered for new architectures. It’s easiest to adopt an immutable data store when most data generated is event oriented. It’s more challenging but still possible to adopt it for use cases traditionally handled by a mutable store such as CRUD of domain objects.
Riak
Riak is a distributed, key value database based on the Amazon Dynamo paper. We have not used Riak in production but have done some extensive testing of it with another development team. We encountered two issues during testing, one of which required a fix to the Riak Java Client. Riak’s support was very helpful in getting the issues resolved even though we weren’t a customer. n We’re impressed with the design and implementation of Riak. It seems to have very good horizontal scaling capabilities. Managing the eventual consistency can be difficult in an application. Using immutability makes this easier.
Assess
GlusterFS
GlusterFS is a distributed file system. It combines storage bricks over Infiniband or TCP/IP into a single networked file system. It was designed to scale across many existing file systems up to several peta-bytes. n
Cross Platform Mobile Frameworks
Tools such as Phonegap and Calatrava offer a way to build a single native application for multiple mobile platforms. They are an alternative to building multiple custom applications for each platform or a single mobile web application. They can give the performance of a native application and the ability to fit within the existing App Store ecosystems. There are many shortcomings and the usage should be evaluated on a case by case basis.
Languages & Frameworks
Adopt
Clojure
Clojure is a functional programming language for the JVM. We’ve found functional programming to be straightforward in Clojure. We’ve been impressed with how useful immutable data structures are. Clojure even offers some of our favorite OO features like polymorphism. Clojure is really compelling from a development perspective. The integration of editor and REPL works really well. Arbitrary blocks of code can be evaluated and run at any time giving immediate feedback.
SASS/Less
SASS and Less are CSS extensions that help keep your CSS organized and gives you helpful tools like variables and mixins. Adopt for large projects only, hand-coded CSS is sufficient for smaller projects.
Reactive Cocoa
Reactive Cocoa brings Functional Reactive Programming to ObjectiveC development. Reactive Cocoa uses continuous streams of input which are transformed into continuous output. Parts of an application are connected together like cells in a spreadsheet. This brings a more functional style to ObjectiveC. The developer describes the desired behavior declaratively instead of how it should be accomplished as with imperative programming. n More information: Big Nerd Ranch video, Overview and example at nshipster.com
Trial
ClojureScript
ClojureScript is a dialect of Clojure that compiles to JavaScript. It has all the benefits of Clojure but applied to a browser environment or even Node.js. The tooling and documentation has some room to grow. Recent developments like the arrival of source maps which allow debugging ClojureScript directly in the browser show a lot of promise.
Assess
Elixir
Elixir is an exciting new dynamic functional programming language built to run on the Erlang VM. It borrows many of the best ideas from other languages like Ruby, Haskell, and Clojure. It provides a Ruby like syntax, immutable data structures, lazy streams, macros, and all the benefits of the Erlang VM like the ability to spawn many light-weight processes that communicate via messaging. Elixir is very young but shows a lot of promise.
ECMAScript 6 (ES6) via the Traceur compiler
ECMAScript is the standard behind JavaScript. The next version, ECMAScript 6 (ES6), adds many new features to the language like default parameter values, lexical block scope with the let keyword, and some new syntax changes for conciseness like the new function syntax.
Traceur is a JavaScript compiler that lets you adopt these future features and compile them to currently supported versions. There’s some risk in adopting a standard before it’s actually accepted. However the ES6 website has a good list of which features have reached consensus and are very likely to be adopted in the next version. Traceur is supported by Grunt and would make an interesting addition to the Rails asset pipeline.
Hold
CoffeeScript for client-side scripts
We recommend carefully considering use of CoffeeScript on new large JavaScript production applications. It looks as though many of its best features, including classes, splats, and destructuring assignments, will be added to ES6. It’s not clear how CoffeeScript will evolve with ES6.
CoffeeScript, unlike ES6, is unlikely to ever be natively supported by browsers. It doesn’t minify well, particularly when including many small files. Its class conventions tend not to play nice with existing libraries, leading to inconsistent coding styles. The style of programming is close enough to JS that it makes sense to just use JS.