Category Archives: Software Engineering

Blockchain — A Definition for Software Engineers

Blockchain has become a buzzword grown out of Bitcoin and subsequent cryptocurrencies. The original Bitcoin paper did not use the term blockchain and it took some time for the term to emerge, in order to describe the underlying technology that permits implementing digital currencies and other applications.

Consequently, and also due to the decentralised nature of blockchain communities, there is no official definition of the term. Many definitions out there are aimed at the investment community or the general public, which is fine, but means they lack a technical depth. Other definitions merely reflect the lack of understanding of the author. Those with a background in Computer Science or Software Engineering that would like to understand the underlying technologies need more precise definition.

Therefore, here is a definition aimed at software engineers and computer scientists:

A blockchain is a linked list data structure, implementing an infinite state engine with immutable state transitions, on top of a peer-to-peer network with a byzantine failure model.

Let’s take a look at this definition one concept at a time.

Linked list data structure

The data held in a blockchain is represented in a list of block, with each block linking back to the previous one. The following diagram is from the original Bitcoin paper and shows how data (transctions) is organised in blocks.


Bitcoin block structure

Blocks are linked by including the cryptographic hash of the previous block. All blockchains have a similar way of organising data. However, just defining a blockchain as a linked list of blocks is not enough. Some additional properties are required.

Infinite state engine

Blockchain systems represent state engines. State engines are systems modelled as a series of state through which the system transitions. State engines are termed finite, if there is a finite number of possible system states. The possible states are known in advance. In the case of infinite state engines, there is an endless number of possible states.


Example of a state engine

Blockchains thus, are infinite state engines. Transactions take the system from one state to another. System state may be account balances, as in Bitcoin’s case, represented by the set of UTXOs (unspent transaction outputs). In more general purpose blockchains, such as Ethereum, state can represent any piece of data.

A virtual machine (VM) typically executes transactions to take the system form one state to another. The Bitcoin VM executes a domain specific script language encoded in transaction inputs and outputs. The Ethereum VM executes more complex operations, as it is touring complete (it can be used execute anything that can be modelled computationally), but the concept is the same.

Transaction immutability

Transactions, which we have just defined as state transitions, are immutable in a blockchain system. This means, that once a transaction has been confirmed, i.e. included in a valid block, it cannot be undone. State can only be reversed by issuing another transaction, but history of transactions cannot be modified.

Peer-to-peer network (P2P)

The data structure and the infinite state engine a system implements is no enough to make it a blockchain. A real blockchain executes on a P2P network. Each full node has a copy of the data-structure, and importantly, all nodes reach consensus on the correct version of the data. That is; all nodes agree on the same system state. A blockchain system thus implements various P2P protocols, such as node and neighbour detection, group communication protocols and consensus protocols.

Byzantine failure model

The biggest achievement of blockchains is probably, that they achieve all the above in a byzantine failure model. In distributed systems research a failure model is the assumption on what may go wrong in a system. For instance, the crash failure model assumes that nodes may crash and the the partition failure model assumes that nodes (or groups of nodes) may be temporarily isolated, due to network faults. Both failure models however assume that nodes try to behave correctly, i.e. there are no malicious participants.

In the byzantine failure model we assume that a node may behave in any way, i.e. system state integrity may not be the goal of every node and there may be malicious participants. Thus, no node is assumed to be trusted.

I was working in distributed systems research in the early 2000s and we managed to implement pretty reliable systems with crash and partition failure models, but byzantine failure models were considered extremely difficult. Blockchains present a working solution to this problem, which is why they can actually be used to model financial transactions in trustless systems.

Blockchains achieve operation with a byzantine failure model by implementing strict consensus mechanisms which include financial incentives for nodes to maintain state integrity, such as proof-of-work or proof-of-stake based models implemented by cryptocurrencies. This also explains why blockchains usually come with their own currency and why transactions tend to have an ascociated transaction fee.

Supporting a byzantine failure model is very costly. This is the main reason why blockchain solutions scale badly. It is important to think carefully wether a problem requires a blockchain based solution, or may work on a traditional centralised system or even a distributed system that does not require a byzantine failure model.

Relaxed definitions and technology advances

The above definition is muddied in some systems which are usually also grouped in the blockchain category. These include systems that relax the failure model by including some trusted nodes, for example systems that use proof-of-authority consensus schemes, in which blocks are created by a set of trusted nodes.

At the other end of the scale there are systems with similar properties that replace the underlying data structure. For example there are systems that build on directed acyclic graphs instead of linked lists of blocks, such as RaiBlocks and IOTA.

Blockchain — A Definition for Software Engineers

Blockchain has become a buzzword grown out of Bitcoin and subsequent cryptocurrencies. The original Bitcoin paper did not use the term blockchain and it took some time for the term to emerge, in order to describe the underlying technology that permits implementing digital currencies and other applications.

Consequently, and also due to the decentralised nature of blockchain communities, there is no official definition of the term. Many definitions out there are aimed at the investment community or the general public, which is fine, but means they lack a technical depth. Other definitions merely reflect the lack of understanding of the author. Those with a background in Computer Science or Software Engineering that would like to understand the underlying technologies need more precise definition.

Therefore, here is a definition aimed at software engineers and computer scientists:

A blockchain is a linked list data structure, implementing an infinite state engine with immutable state transitions, on top of a peer-to-peer network with a byzantine failure model.

Let’s take a look at this definition one concept at a time.

Linked list data structure

The data held in a blockchain is represented in a list of block, with each block linking back to the previous one. The following diagram is from the original Bitcoin paper and shows how data (transctions) is organised in blocks.


Bitcoin block structure

Blocks are linked by including the cryptographic hash of the previous block. All blockchains have a similar way of organising data. However, just defining a blockchain as a linked list of blocks is not enough. Some additional properties are required.

Infinite state engine

Blockchain systems represent state engines. State engines are systems modelled as a series of state through which the system transitions. State engines are termed finite, if there is a finite number of possible system states. The possible states are known in advance. In the case of infinite state engines, there is an endless number of possible states.


Example of a state engine

Blockchains thus, are infinite state engines. Transactions take the system from one state to another. System state may be account balances, as in Bitcoin’s case, represented by the set of UTXOs (unspent transaction outputs). In more general purpose blockchains, such as Ethereum, state can represent any piece of data.

A virtual machine (VM) typically executes transactions to take the system form one state to another. The Bitcoin VM executes a domain specific script language encoded in transaction inputs and outputs. The Ethereum VM executes more complex operations, as it is touring complete (it can be used execute anything that can be modelled computationally), but the concept is the same.

Transaction immutability

Transactions, which we have just defined as state transitions, are immutable in a blockchain system. This means, that once a transaction has been confirmed, i.e. included in a valid block, it cannot be undone. State can only be reversed by issuing another transaction, but history of transactions cannot be modified.

Peer-to-peer network (P2P)

The data structure and the infinite state engine a system implements is no enough to make it a blockchain. A real blockchain executes on a P2P network. Each full node has a copy of the data-structure, and importantly, all nodes reach consensus on the correct version of the data. That is; all nodes agree on the same system state. A blockchain system thus implements various P2P protocols, such as node and neighbour detection, group communication protocols and consensus protocols.

Byzantine failure model

The biggest achievement of blockchains is probably, that they achieve all the above in a byzantine failure model. In distributed systems research a failure model is the assumption on what may go wrong in a system. For instance, the crash failure model assumes that nodes may crash and the the partition failure model assumes that nodes (or groups of nodes) may be temporarily isolated, due to network faults. Both failure models however assume that nodes try to behave correctly, i.e. there are no malicious participants.

In the byzantine failure model we assume that a node may behave in any way, i.e. system state integrity may not be the goal of every node and there may be malicious participants. Thus, no node is assumed to be trusted.

I was working in distributed systems research in the early 2000s and we managed to implement pretty reliable systems with crash and partition failure models, but byzantine failure models were considered extremely difficult. Blockchains present a working solution to this problem, which is why they can actually be used to model financial transactions in trustless systems.

Blockchains achieve operation with a byzantine failure model by implementing strict consensus mechanisms which include financial incentives for nodes to maintain state integrity, such as proof-of-work or proof-of-stake based models implemented by cryptocurrencies. This also explains why blockchains usually come with their own currency and why transactions tend to have an ascociated transaction fee.

Supporting a byzantine failure model is very costly. This is the main reason why blockchain solutions scale badly. It is important to think carefully wether a problem requires a blockchain based solution, or may work on a traditional centralised system or even a distributed system that does not require a byzantine failure model.

Relaxed definitions and technology advances

The above definition is muddied in some systems which are usually also grouped in the blockchain category. These include systems that relax the failure model by including some trusted nodes, for example systems that use proof-of-authority consensus schemes, in which blocks are created by a set of trusted nodes.

At the other end of the scale there are systems with similar properties that replace the underlying data structure. For example there are systems that build on directed acyclic graphs instead of linked lists of blocks, such as RaiBlocks and IOTA.

Introducing Project ONYX

Distributed Test Driven Development powered by Ethereum

We are happy to announce Project ONYX, the first decentralized platform to enable Distributed Test Driven Development built on the Ethereum blockchain.

Learn more at https://projectonyx.io where you can find our whitepaper and our alpha build. All code is open source and is available at our Github. Come join us on Slack, Telegram, Twitter, or Reddit!

Distributed Test Driven Development (d-TDD)

Test Driven Development (TDD) is a development practice that focuses on making software by planning and specifying expected behaviors with tests before coding up a solution. These tests fully lay out the intended behavior of the software the company is trying to create and any code (implementation) that passes these tests is deemed to be correct. When adding a new feature the following loop is applied:

  1. Add new tests that the new feature will have to pass
  2. Confirm the current code passes old tests and fails new tests
  3. Implement new feature so all of the tests pass
  4. Confirm all of the tests pass again

The benefits of TDD include fewer errors in code and better code and overall software quality when compared to industry standard. These benefits are mainly because TDD leads to better testing coverage over a codebase and more modular and testable software architectures.

Even though TDD offers these great advantages, it has drawbacks that hinder it from adoption in many spaces. These drawbacks are that it takes more employees to handle the extra work from the extra testing on top of the implementation (more than industry average) and that it can potentially take longer to implement features (but they will be of better quality).

With these current limitations in mind, we present the practice of distributed TDD (d-TDD) enabled by Project ONYX. A company practicing d-TDD should receive all of the benefits of TDD while avoiding (or severely reducing) the drawbacks.

With Project ONYX, companies can simply write a test suite to specify the behaviors they want in the code (step 1 of the loop) and let engineers on the ONYX network handle the implementation work (step 3 of the loop). All work is guaranteed to pass the test suite by a network of Validators and all transactions are guaranteed by smart contracts on the Ethereum blockchain. By leveraging d-TDD, building good quality code takes fewer engineers and can be done at the same velocity or potentially even faster than traditional methods.

The world runs on software. Everything from your car to your home now has software embedded into it to make it smarter and more useful. While this boom has caused huge changes in almost every industry in the world, the way software is developed remains about the same.

Why do things need to change?

In today’s environment, a company will hire many software engineers to design and implement a system. The company buys the engineers’ time and the engineers work solely for the company. However, this model has some flaws and inefficiencies that could be improved to lead to even more cost savings and convenience for all parties.

In the current model, any idle time that an engineer has working for a company is paid for by the company. It can be seen as the cost for having an engineer handle work at a moments notice. From industry research and personal experience, this idle time is not negligible and can account for almost 30–40% of an engineers time. Not only are engineers idle, many times seasoned engineers have to handle menial development tasks that may be better suited to newer, cheaper engineers.

Both of these issues together point to under-utilization of great engineers. This is detrimental to everyone involved; engineers get bored and companies have to spend more money. We present a better way with Project ONYX.

How does Project ONYX help?

We enable companies to hire fewer high quality engineers to design the software, write the tests, and review code while outsourcing implementation to ONYX. Companies don’t need to have more engineers to handle implementation work at a moments notice because they have a completely scalable supply of engineers ready to do their work on the ONYX network. This is essentially engineering supply on demand, saving companies both time and money.

On the other hand, engineers on the ONYX network get to pick tasks they are more passionate about or have the skill set to easily solve. Considering many companies have similar problems that they need to solve for their products, engineers can even reuse code for different tasks to collect more money for the same amount of work! Also, engineers are not bound by long term commitments to any one company and all work can be done on one’s own schedule and preferred location.

How does Project ONYX work?

ONYX has 3 main players on the network:

  1. Requesters: Place the request for tasks with test suites, starter code, and payment for the work (mostly software development companies).
  2. Engineers: People on the network who want to provide engineering work for money.
  3. Validators: People who will run a automatic validation script that will listen to the blockchain to receive code from engineers and let’s the parties know if the tests pass for a small cut of the transaction (anybody with a computer).

Each of these players has something they offer and something they need. They use Project ONYX to facilitate all of these transactions. The company will take a small cut from the transactions as well to fund development and improvement.

Why use the Ethereum blockchain?

The peer-to-peer nature of d-TDD is an excellent use case for the Ethereum blockchain. Ethereum offers smart contracts allowing us to write code to implement the rules for the network. All parties involved can easily verify these rules by reading the code and can guarantee that all attending parties will follow them without the need for trust or a 3rd party. What this means in practice is that nobody has to worry that they won’t be paid or get the work that their owed.

Also, with Ethereum behaving as our backend, everyone interacts with the blockchain instead of our own servers which makes our solution much easier to scale. In addition to this, by using IPFS for file storage and encryption, it makes it so that we as a company don’t hold any code and the protocol is truly secure and p2p.

ONYX Tokens

An integral component to the network we describe is the ONYX token we will be releasing. The purpose of the ONYX tokens are to behave as a required stake for any transaction. What this means is that for any player in the network to participate, they have to stake a certain number of ONYX tokens for the transaction (Note that this is for each transaction so companies will require multiples of this stake to request multiple tasks). The stake makes it so that any party is motivated to see the ONYX platform succeed and any malicious actors can be penalized by losing their stake. No value will be transferred using ONYX tokens. In effect, the ONYX tokens truly behave as a membership token into using the network.

The amount of stake necessary to participate is set by ONYX token holders by a voting scheme. A dynamic stake is necessary because ONYX token prices aren’t expected to be static and the amount of stake required affects all participants. A higher stake will mean only larger players can afford to participate and lead to higher quality of users while a lower stake will allow more players to participate but may lead to lower quality users. This is a decision that directly impacts all token holders so we leave setting that stake amount to them.

More concrete details for this token scheme can be found in the whitepaper.

How are we different from freelance work?

The non-employment based work may remind many of the current freelance market products like Upwork and even other products that are currently built on Ethereum like Ethlance. However, we offer many benefits over the current paradigm.

Current problems in freelance:

  • Unclear goals from companies that can lead to stress for freelancers
  • Potentially unclear duration of commitment

With Project ONYX, we solve these issues inherently by the protocol:

  • All tasks requested come with a very specific test suite to pass which erases all ambiguity of what is required
  • Since engineers can browse the tasks before claiming them, they have a better idea of how long implementation work will take and clear expectations by deadlines set by Requesters

Truly nothing exists currently that solves the problem we are tackling in the way we are solving it and we are the first to put focus on TDD practices.

Development Notes

We have currently finished building an alpha that already showcases many of the features we have described. All the code is already open sourced and can be found at our Github. Documentation is still being heavily written so the codebase and clarity will improve over time.

Feedback

We would love to hear feedback from the community and continue to improve the product to match all expectations. Come join us on slack for any questions or concerns you may have or comment below.

Thanks for reading and we look forward to any and all feedback!

Introducing Project ONYX

Distributed Test Driven Development powered by Ethereum

We are happy to announce Project ONYX, the first decentralized platform to enable Distributed Test Driven Development built on the Ethereum blockchain.

Learn more at https://projectonyx.io where you can find our whitepaper and our alpha build. All code is open source and is available at our Github. Come join us on Slack, Telegram, Twitter, or Reddit!

Distributed Test Driven Development (d-TDD)

Test Driven Development (TDD) is a development practice that focuses on making software by planning and specifying expected behaviors with tests before coding up a solution. These tests fully lay out the intended behavior of the software the company is trying to create and any code (implementation) that passes these tests is deemed to be correct. When adding a new feature the following loop is applied:

  1. Add new tests that the new feature will have to pass
  2. Confirm the current code passes old tests and fails new tests
  3. Implement new feature so all of the tests pass
  4. Confirm all of the tests pass again

The benefits of TDD include fewer errors in code and better code and overall software quality when compared to industry standard. These benefits are mainly because TDD leads to better testing coverage over a codebase and more modular and testable software architectures.

Even though TDD offers these great advantages, it has drawbacks that hinder it from adoption in many spaces. These drawbacks are that it takes more employees to handle the extra work from the extra testing on top of the implementation (more than industry average) and that it can potentially take longer to implement features (but they will be of better quality).

With these current limitations in mind, we present the practice of distributed TDD (d-TDD) enabled by Project ONYX. A company practicing d-TDD should receive all of the benefits of TDD while avoiding (or severely reducing) the drawbacks.

With Project ONYX, companies can simply write a test suite to specify the behaviors they want in the code (step 1 of the loop) and let engineers on the ONYX network handle the implementation work (step 3 of the loop). All work is guaranteed to pass the test suite by a network of Validators and all transactions are guaranteed by smart contracts on the Ethereum blockchain. By leveraging d-TDD, building good quality code takes fewer engineers and can be done at the same velocity or potentially even faster than traditional methods.

The world runs on software. Everything from your car to your home now has software embedded into it to make it smarter and more useful. While this boom has caused huge changes in almost every industry in the world, the way software is developed remains about the same.

Why do things need to change?

In today’s environment, a company will hire many software engineers to design and implement a system. The company buys the engineers’ time and the engineers work solely for the company. However, this model has some flaws and inefficiencies that could be improved to lead to even more cost savings and convenience for all parties.

In the current model, any idle time that an engineer has working for a company is paid for by the company. It can be seen as the cost for having an engineer handle work at a moments notice. From industry research and personal experience, this idle time is not negligible and can account for almost 30–40% of an engineers time. Not only are engineers idle, many times seasoned engineers have to handle menial development tasks that may be better suited to newer, cheaper engineers.

Both of these issues together point to under-utilization of great engineers. This is detrimental to everyone involved; engineers get bored and companies have to spend more money. We present a better way with Project ONYX.

How does Project ONYX help?

We enable companies to hire fewer high quality engineers to design the software, write the tests, and review code while outsourcing implementation to ONYX. Companies don’t need to have more engineers to handle implementation work at a moments notice because they have a completely scalable supply of engineers ready to do their work on the ONYX network. This is essentially engineering supply on demand, saving companies both time and money.

On the other hand, engineers on the ONYX network get to pick tasks they are more passionate about or have the skill set to easily solve. Considering many companies have similar problems that they need to solve for their products, engineers can even reuse code for different tasks to collect more money for the same amount of work! Also, engineers are not bound by long term commitments to any one company and all work can be done on one’s own schedule and preferred location.

How does Project ONYX work?

ONYX has 3 main players on the network:

  1. Requesters: Place the request for tasks with test suites, starter code, and payment for the work (mostly software development companies).
  2. Engineers: People on the network who want to provide engineering work for money.
  3. Validators: People who will run a automatic validation script that will listen to the blockchain to receive code from engineers and let’s the parties know if the tests pass for a small cut of the transaction (anybody with a computer).

Each of these players has something they offer and something they need. They use Project ONYX to facilitate all of these transactions. The company will take a small cut from the transactions as well to fund development and improvement.

Why use the Ethereum blockchain?

The peer-to-peer nature of d-TDD is an excellent use case for the Ethereum blockchain. Ethereum offers smart contracts allowing us to write code to implement the rules for the network. All parties involved can easily verify these rules by reading the code and can guarantee that all attending parties will follow them without the need for trust or a 3rd party. What this means in practice is that nobody has to worry that they won’t be paid or get the work that their owed.

Also, with Ethereum behaving as our backend, everyone interacts with the blockchain instead of our own servers which makes our solution much easier to scale. In addition to this, by using IPFS for file storage and encryption, it makes it so that we as a company don’t hold any code and the protocol is truly secure and p2p.

ONYX Tokens

An integral component to the network we describe is the ONYX token we will be releasing. The purpose of the ONYX tokens are to behave as a required stake for any transaction. What this means is that for any player in the network to participate, they have to stake a certain number of ONYX tokens for the transaction (Note that this is for each transaction so companies will require multiples of this stake to request multiple tasks). The stake makes it so that any party is motivated to see the ONYX platform succeed and any malicious actors can be penalized by losing their stake. No value will be transferred using ONYX tokens. In effect, the ONYX tokens truly behave as a membership token into using the network.

The amount of stake necessary to participate is set by ONYX token holders by a voting scheme. A dynamic stake is necessary because ONYX token prices aren’t expected to be static and the amount of stake required affects all participants. A higher stake will mean only larger players can afford to participate and lead to higher quality of users while a lower stake will allow more players to participate but may lead to lower quality users. This is a decision that directly impacts all token holders so we leave setting that stake amount to them.

More concrete details for this token scheme can be found in the whitepaper.

How are we different from freelance work?

The non-employment based work may remind many of the current freelance market products like Upwork and even other products that are currently built on Ethereum like Ethlance. However, we offer many benefits over the current paradigm.

Current problems in freelance:

  • Unclear goals from companies that can lead to stress for freelancers
  • Potentially unclear duration of commitment

With Project ONYX, we solve these issues inherently by the protocol:

  • All tasks requested come with a very specific test suite to pass which erases all ambiguity of what is required
  • Since engineers can browse the tasks before claiming them, they have a better idea of how long implementation work will take and clear expectations by deadlines set by Requesters

Truly nothing exists currently that solves the problem we are tackling in the way we are solving it and we are the first to put focus on TDD practices.

Development Notes

We have currently finished building an alpha that already showcases many of the features we have described. All the code is already open sourced and can be found at our Github. Documentation is still being heavily written so the codebase and clarity will improve over time.

Feedback

We would love to hear feedback from the community and continue to improve the product to match all expectations. Come join us on slack for any questions or concerns you may have or comment below.

Thanks for reading and we look forward to any and all feedback!

Visa is looking for Blockchain Developers

Visa

March 2, 2016 — Visa is looking for Blockchain developers. In a recent job posting on Visa’s website, the payment processor noted that their software engineers are tasked with:

“Designing and developing secure, scalable blockchain network and micro services that are highly scalable, efficient, and extensible.”

Read also: Wall of Coins Review: Good for Bitcoin Beginners, So-So for Veterans

Visa Looks to Blockchain for Future Infrastructure

 

The list of qualifications seems to have a consistent bias towards skills that lend themselves to blockchain application development as well. Whether they plan on an internal infrastructure update or rolling out a retail product based on blockchain technology is up in the air, but the posting displays a novel seriousness on Visa’s part about the potential of enterprise blockchain development.

Bitcoin adoption has been enjoying explosive growth in places where the value of local currency is dropping, and crypto-based payment applications have become increasingly viable as efficient alternatives to services like Apple Pay and Visa Checkout. With the recent release of IBM’s OpenBlockchain and government and banks looking extensively into blockchain development, it comes as no surprise that Visa shares a belief in the blockchain’s place in modern finance. With Apple Pay adoption stagnating and a lack of a universal solution for convenient payment, Cryptocurrency becomes an attractive option, hence recently increased adoption. Visa and other companies may well need to take this route to remain viable in the current Fintech ecosystem.

This development from Visa contrasts directly with attitudes towards blockchain development just a few years ago on the enterprise side.  Open source solutions to finance used to be regarded as Obtuse and impractical, and public perception was hardline negative. Now, though, Visa is looking to develop blockchain software and technology. More than anything, this move is an acceptance of the reality that decentralized finance being the norm is becoming a distinct possibility. A possibility financial institutions and technology companies like Visa are going to have to deal with in the near future to continue doing business.

What do you think Visa’s search for blockchain devs will produce? Let us know in the Comments!


Images courtesy of Visa, Apple

The post Visa is looking for Blockchain Developers appeared first on Bitcoinist.net.