Overlock runs a small open-source daemon process which local programs can log to (over a unix domain socket). This allows programs to store state and log messages (bread crumbs) as well as reporting exceptions in code. Depending on configuration and rules, the Overlock Daemon stores data for a period of time, either in memory or a file, so that it can be retrieved or reported if there is a problem.
Each Overlock daemon opens an MQTT(S) connection to Overlock in order to receive commands to retrieve logging information if there is a problem elsewhere in the system. The MQTT(S) connection is a low power, low data transfer connection which overall is much cheaper than HTTPS.
Sentry, Rollbar and co are great for catching errors in relatively self contained pieces of code - e.g. an app, web-server, where the interfaces to other sections of code are stateful (read RESTful), however when there is state spanning many separate devices, existing solutions are not able to correlate the state from disparate sources when a problem occurs.
Overlock builds on the concepts of these systems, adding the ability to get info from all parts of your system when something goes wrong, leading to a much faster resolution to problems.
Text logging is a great tool and something all of the team at Overlock have used on a number of IoT deployments, however we’ve commonly found that more than 5X as much data is transferred for logging than for the actual function of the device, which makes it a disproportionate fraction of the data costs for cellular based systems and can even cause problems for users on low cost broadband connections for devices in the home.
From previous IoT deployments, we’ve found searching through log data from devices, gateways and servers to be cumbersome. The alternative is to log virtually everything that happens and even this often does not contain enough information to identify the root cause of issues.
Many deployments we’ve seen avoid the costs of hosted logging solutions by storing the logs on the edge devices and simply remotely accessing the logs (through SSH/remote desktop).
This can be troublesome for 2 reasons:
a) It’s a purely reactive approach. Problems are only brought to light by the customer and a login must be carried out to investigate the issue.
b) It necessitates the use of remote access, which creates a much larger attack surface on the IoT device
Overlock receives application data which has been added to the state by a developer. This data is then transferred to Overlock using a secure channel (MQTTS), which ensures your data is safe. The only point of contact with your device is the MQTT connection, which is hard-coded to connect to the Overlock servers and does not open any ports on your devices or servers.
Overlock supports all linux based platforms, which according to our research is 44% of IoT devices, 67% of IoT gateways and virtually all servers.
Currently Overlock supports C and Python, however we’re also looking at others such as Java. If you’d like to see support for a particular language, contact us here, or have a look at the API docs for the Overlock daemon to see how to create a binding.
Overlock is compatible with all linux devices and platforms where you can add exception handlers to your code and track application state.
Overlock runs on a multi-tenant cloud system for regular customers, however we have the ability to host dedicated instances in any Kubernetes cluster, which will be available through our cloud. In short, yes.
Overlock is being used on live customer projects by Zoetrope labs (the creators of Overlock), as well as with a limited group of companies as part of our private Beta program.
Overlock has saved us a great deal of time in reaching “ah-ha!” moments when searching for obscure (and retrospectively obvious) bugs in development, pilot and production systems. Whereas it could have taken us days to find and reproduce problems, Overlock provides detailed insight taking us straight to the issue.
Currently the Overlock daemon only runs on Linux, however we’d love to hear from you if you’re looking to debug embedded devices.
Overlock is refining the system and getting feedback from a limited group of teams at the moment. We’re gradually rolling it out to more teams and will have a public release during Q1 2018.