Overview

Our service development work is done by a team divided between Tokyo and Osaka, working together with a startup.

While it is impossible for the two groups to have the kind of easy conversations that are possible when everyone is on the same site, real-time communication is essential for working as a team on development work.

When carrying out PoC (proof of concept) work for service development, it was necessary to access the API from smartphones and IoT devices through the Internet, which required a widely accessible execution environment.

To accomplish this, we used a cloud service to build a development environment that allows real-time communication that is not contingent on location and an execution environment that is widely accessible for PoC.

The development environment is accessed by VPN to control access and prevent intruders.

These documents explain the specific ways in which our development environment stack and execution environment stack are used.

Overall configuration of environments

Below is a map of the environment we use.

There are two VPCs on the AWS, a development environment VPC and an execution environment VPC.
The two VPCs are connected by VPC peering.

Connections can only be made from the development environment VPC to the execution environment VPC, not the reverse direction.
This is to prevent situations where an outside party exploits a vulnerability in the application installed in the execution environment, enters the development environment and acquires information such as source code.

The development environment VPC has a public subnet and a private subnet. A product used to make VPN connections is installed in the public subnet. The private subnet contains development tools such as JIRA and GitLab, along with a jump server.
Information on the development tools will be provided later. The jump server is installed to enable access to the execution environment using SSH.

There is a private subnet in the execution environment VPC, and a web API is installed to enable access for purposes such as PoCs.
The web API is accessed for maintenance via the jump server in the development environment VPC.
API Gateway is used to access the web API via the internet using HTTP.

When performing PoCs, HTTP communication is performed over the Internet using a smartphone or IoT device.
Authentication with an API key is performed by API Gateway in addition to the authentication of the application to prevent unauthorized access by outside parties exploiting any vulnerability in the application installed in the execution environment.

About our development and execution environment stacks

Development environment stack

This document provides a basic description of the tools we use for team development when the team is divided between multiple locations.

In addition to ensuring that the functions of each tool are sufficient, we needed to make sure that each tool could be installed in our environment, as some tools need to be used via a VPN to control access and prevent intruders.

JIRA

JIRA is project management software provided by Atlassian.

We use it to manage backlog and share information for Scrum events by displaying it on our Kanban.

While JIRA is also provided as a service, we have installed the server version on our private subnet, which can be accessed by VPN.

GitLab

A version management system.

GitLab is also provided as a service, but we have installed it on our private subnet, which can be accessed by VPN, just like JIRA.

GitLab has a CI function in addition to its version management function. We use the CI function to build source code, which we then deploy in Nexus (described in the next section).

Nexus

This software can be used to build private Maven repositories and Docker registries.

We mainly use it as a place to deploy deliverables.

Like JIRA and Gitlab, we have installed Nexus on our private subnet, which can be accessed by VPN.

Zoom

This service enables video calls using computers or smartphones.

Zoom brings the whole team together for discussions during Scrum events. It is easy to use and both the picture and audio are of high quality, enabling stress-free video calls.

Slack

A chat service.

This is the main tool we use for daily communication.

Execution environment stack

API Gateway

This service is provided by AWS.

While each of the services we develop has its own authentication process, API Gateway guarantees security regardless of the development status of the service as it enables authentication with an API key.

HTTP requests sent from smartphones and IoT devices pass through API Gateway before reaching the service on the private subnet in our execution environment. The services vary: some are built using Spring Boot and executed using java commands while others are built using Ruby on Rails and executed using Docker.

How our development environment stack is used in Scrum development

This document explains how each of the tools in our development environment stack is used for Scrum events and day-to-day activities in development by a Scrum team that is divided between multiple locations.

This table provides an overview of the main tools used for each Scrum event and daily activity.

JIRA GitLab Nexus Slack Zoom
Sprint planning X X X
Daily scrums X X
Sprint reviews X X
Retrospectives X X
Implementation X X
Reviews X
Documentation X
Daily communication X X

Sprint planning

Backlog

  • Tools used: JIRA, Zoom

We use JIRA for backlog management.

When the whole team works in the same place, it is probably common for the team members to meet in a meeting area and look at the same screen while backlog is shared.

Even when team members are not working in the same location, Zoom enables the whole team to share the same screen when the backlog or tasks are being shared among the team members.

  • Example of backlog screen

Planning poker

  • Tools used: Slack, Zoom

Planning poker is used to estimate the working time of product backlog items.

Planning poker is a method where team members use cards with numbers such as 1, 2, 3 and 5 to estimate the relative size of each task.

When we worked on projects where the whole team was in one location, the team members showed their cards based on how large they thought each task was.

A similar method can be used for teams divided between multiple locations: the title of a product backlog item is copied to Slack and members then post number reactions at the same time in a Zoom conversation after being prompted.

ThiThis provides a visual representation where everyone can see at a glance which number was most commonly posted, so this can probably be used just the same for physically separated teams as for those who are all in the same place.

  • An example of planning poker in Slack

Daily scrums

  • Tools used: JIRA, Zoom

Zoom is used to bring the team members together for daily scrums, where they share the same screen and discuss the necessary points in a video call.

During daily scrums, the team looks at the tasks on JIRA and each team member shares the following information.

  • What they did the previous day to contribute to the development team achieving the sprint goals
  • What they will do in the day ahead to contribute to the development team achieving the sprint goals
  • Any faults they encountered that may make it difficult for themselves or the development team to achieve the sprint goals

Reference:

Sprint reviews

  • Tools used: JIRA, Zoom

Zoom is used to bring the team members together. The team members provide reports and demonstrations of the current status of the sprint while looking at the backlog in JIRA.

When demonstrating using actual devices such as Android or iPhone, be aware that members at other locations can also see it.

Retrospectives

  • Tools used: Slack, Zoom

KPT (Keep, Problem, Try) is used for retrospectives.

“Keep” indicates points that went well and should be continued. “Problem” indicates points that went poorly or posed issues. “Try” indicates points to be tried next.

Reference: Retrospective Guide (external website, only Japanese edition)

When we worked on projects where the whole team was in one location, KPT was mapped with sticky notes on simili paper.
Each team member wrote their “Keep” points on a sticky note and then shared them, and did the same for each “Problem” and “Try”.

With physically separated teams, a similar process can be performed using Slack and Zoom.

We use a KPT channel on Slack to make it easy to refer to the discussion again later.
Points are shared in order of “Keep, Problem, Try” in the KPT channel.

Team members can use stamps to react to comments they agree with.
This makes it easier to get reactions from team members who have difficulty speaking up.

  • Example of KPT on Slack

Implementation

CI

  • Tool used: GitLab（GitLab CI）

GitLab CI is used to automatically execute build and test when each developer commits.

CI also runs automatically when a merge request is created, enabling team members to instantly see if there are any build or test failures and thereby reducing unnecessary work in reviews.

The physical separation of the team makes it all the more important to constantly monitor the source code for bugs and other issues using CI to reduce unnecessary communication costs that occur in cases such as those where a fix at the other location delays development.

Sharing of deliverables

  • Tool used: Nexus

Nexus is used to manage libraries and deliverables for a project.

Installing Nexus in VPN-protected environments makes it possible to share deliverables safely via the VPN when the team is physically separated.

Reviews

  • Tool used: GitLab（Merge Request）

Reviews are performed using GitLab’s Merge Request.

  • Example: Merge Request screen

The reviewee creates the branch, fixes the source code and submits a review request using Merge Request in GitLab.

The reviewer checks the Merge Request and writes a comment in GitLab if they have any concerns.

The review conversation is conducted entirely in GitLab, enabling a review to be conducted even if the reviewer and reviewee are in different locations.

  • Example: Comments for a review

Documentation

  • Tool used: GitLab（Wiki）

Documents are shared using GitLab’s Wiki function.

This is mainly used to keep information such as the environment building procedure or tips that should be shared with the team.

This is convenient as it enables linking of procedures and documents among a physically separated team, simply by posting the URL in the Wiki to Slack.

  • Example: Wiki screen

Daily communication

  • Tools used: Slack, Zoom

Slack and Zoom are used for general communication.

Team members communicate on Slack using channels such as the team channel and their own monologue channel.

Monologue channels are channels that each member sets up for themselves. The channels must be open so that anyone can read them, and each team member must make as-needed posts in their own channel about points such as what they are about to do, impressions of work that they have done and any difficult points that they are struggling with.

This allows each team member to be seen and heard by the others, and speaking up about difficult points enables help to be given by team members who know a solution.

  • Example: A monologue channel

Since team members do not know when their teammates at the other location start and finish work, we set up an attendance channel so that team members can indicate when they start and finish work and when they take holidays.

  • Example: Attendance channel

Zoom is used for Scrum events, basic meetings and pair programming.
Zoom enables smooth communication among physically separated teams as users can share screens, make video calls and even share whiteboards.

These documents are provided under the international Creative Commons Attribution-ShareAlike 4.0 license.