Skip to content

Dev docker wrapper #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
spamoom wants to merge 27 commits into
base: master
Choose a base branch
from
Open

Dev docker wrapper #7

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
3affc5b
Initial commit
spamoom May 18, 2021
ab75c70
Pointing CLI to docker wrapper
spamoom May 18, 2021
d6579ab
Fixes bug introduced by pulling docker compose in
spamoom May 20, 2021
5891bd9
Improves shell script to allow for better updating
spamoom May 20, 2021
5a03335
Better self-update text
spamoom May 20, 2021
25a6ea5
Moving aws to readonly and removes ssh
spamoom May 20, 2021
5b3db23
Using system temp directory so we're still compatible with desktop users
spamoom May 20, 2021
73fe766
Removes bleed commit from other work
spamoom May 20, 2021
19062e1
Ensuring the wrapper uses latest
spamoom May 20, 2021
41b12f2
Further WIP bleeding!
spamoom May 21, 2021
60235f2
Adds more directories to dockerignore
spamoom May 21, 2021
2cfef32
More consistent naming
spamoom May 21, 2021
524b980
Update app/Commands/AwsSsmConnect.php
spamoom May 21, 2021
bd817a8
Adds support for SSH tunnels and switches wrapper to sh
spamoom May 21, 2021
d3dba73
Merge branch 'dev-docker-wrapper' of github.com:netsells/cli into dev…
spamoom May 21, 2021
55a67a0
Reverts to docker-compose
spamoom May 21, 2021
5803677
Installs docker-compose
spamoom May 21, 2021
f9a861b
Adds support for using AWS cli to auth docker
spamoom May 21, 2021
83504e7
Further shell script support
spamoom May 21, 2021
115beec
Remove weird spaces
May 21, 2021
469abf1
Optimise dockerfile to reduce the final image size.
May 22, 2021
7a68af3
Add installation instructions for the docker wrapper.
May 22, 2021
a0cf44a
Only run docker login, if the ecr credential helper is not present.
May 22, 2021
0eb014c
Fix credential helper error
May 22, 2021
e6f3769
Add local docker build instruction.
May 22, 2021
a7877f7
Ran composer update, as the lock file was out of sync.
May 22, 2021
479cacc
Reverts ECR auth to standard docker login
spamoom May 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
vendor/
Comment thread
spamoom marked this conversation as resolved.
56 changes: 56 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
##
# Composer
##
FROM composer:latest as composer

##
# PHP Builder
##
FROM php:7.4-cli as build

ARG DOCKER_TAG

# Deps
RUN apt-get update && apt-get install -y \
unzip \
git

# Grab composer
COPY --from=composer /usr/bin/composer /usr/bin/composer

COPY . /app
WORKDIR /app

# Install CLI deps
RUN composer install --ansi --no-interaction --no-progress --prefer-dist

# Build the phar
RUN php netsells app:build --build-version=$DOCKER_TAG

##
# PHP Runtime
##
FROM php:7.4-cli as runtime
Copy link
Copy Markdown
Contributor

@kubatek94 kubatek94 May 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can do few tweaks here to make the resulting image considerably smaller, and also to make the self-update pull down only the things that changed. Are you ok for me to look into it later on in the evening?

Copy link
Copy Markdown
Contributor Author

@spamoom spamoom May 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely mate - I was doing this between meetings so it's very WIP - just wanted it in front of you as I knew you'd spot a load of improvements

Copy link
Copy Markdown
Contributor

@kubatek94 kubatek94 May 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, will try few ideas and report back :)

Copy link
Copy Markdown
Contributor

@kubatek94 kubatek94 May 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've done few tweaks to reduce the image by > 300MB. It's still big, but at least it has all tools inside it which is convenient.

ECR helper works well after a fix which is great, however worth testing again on TeamCity, as it won't have access to the AWS config file I think?

I do have problems using the SSM connect feature, but not sure if it's just all our EC2 instances not having it enabled or something? Would be nice if you can confirm it's working.

Copy link
Copy Markdown
Contributor Author

@spamoom spamoom May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work - yeah the NS EC2 images aren't setup for SSM

Copy link
Copy Markdown
Contributor Author

@spamoom spamoom May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already checked on the YPS Teamcity agent, it's able to access the meta domain so is able to get auth 🌮

http://169.254.169.254/latest/meta-data/


# Grab built phar from the builder
COPY --from=build /app/builds/netsells /usr/local/bin/netsells

# Copy the wrapper from source
COPY ./docker-support/netsells /usr/local/bin/netsells-wrapper

# Deps
RUN apt-get update && apt-get install -y \
unzip \
git \
docker.io

# AWS CLI
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && unzip awscliv2.zip && ./aws/install && rm awscliv2.zip

# Session Manager
RUN curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/ubuntu_64bit/session-manager-plugin.deb" -o "session-manager-plugin.deb" && dpkg -i session-manager-plugin.deb && rm session-manager-plugin.deb

RUN mkdir /app
WORKDIR /app

ENTRYPOINT ["netsells"]
18 changes: 13 additions & 5 deletions app/Commands/AwsSsmConnect.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ class AwsSsmConnect extends Command
*/
protected $description = 'Connect to an server via SSH (Use --tunnel to establish an SSH tunnel)';

protected $tempKeyName = 'netsells-cli-ssm-ssh-tmp';
protected $tempKeyName;

/** @var Helpers $helpers */
protected $helpers;
Expand All @@ -48,6 +48,15 @@ public function configure()
], $this->helpers->aws()->commonConsoleOptions()));
}

private function tempIdentityFile(): string
{
if (!$this->tempKeyName) {
Comment thread
spamoom marked this conversation as resolved.
Outdated
$this->tempKeyName = tempnam(sys_get_temp_dir(), 'NetsellsCliSsm');
}

return $this->tempKeyName;
}

/**
* Execute the console command.
*
Expand Down Expand Up @@ -95,7 +104,7 @@ public function handle()
$sessionCommandString = implode(' ', $sessionCommand->getArguments());

$options = [
'-o', 'IdentityFile ~/.ssh/netsells-cli-ssm-ssh-tmp',
'-o', 'IdentityFile ' . $this->tempIdentityFile(),
'-o', 'IdentitiesOnly yes',
'-o', 'GSSAPIAuthentication no',
'-o', 'PasswordAuthentication no',
Expand Down Expand Up @@ -215,8 +224,7 @@ private function generateTempSshKey()
return 1;
}

$sshDir = $_SERVER['HOME'] . '/.ssh/';
$keyName = $sshDir . $this->tempKeyName;
$keyName = $this->tempIdentityFile();
$pubKeyName = "{$keyName}.pub";

if (file_exists($keyName)) {
Expand All @@ -234,7 +242,7 @@ private function generateTempSshKey()
'-t', 'ed25519',
'-N', "",
Comment thread
spamoom marked this conversation as resolved.
Outdated
'-f', $keyName,
'-C', "netsells-cli-ssm-ssh-session"
'-C', $this->tempIdentityFile()
])
->run();
} catch (ProcessFailed $e) {
Expand Down
6 changes: 3 additions & 3 deletions app/Commands/DeployEcsServiceUpdate.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ public function configure()
*/
public function handle()
{
$requiredBinaries = ['aws', 'docker-compose'];
$requiredBinaries = ['aws', 'docker'];

if ($this->helpers->checks()->checkAndReportMissingBinaries($this, $requiredBinaries)) {
return 1;
Expand Down Expand Up @@ -230,7 +230,7 @@ protected function gatherTargetImages(): array
try {
$dockerComposeConfig = Yaml::parse($dockerComposeYml);
} catch (ParseException $exception) {
$this->error("Failed to parse yml from docker-compose output.");
$this->error("Failed to parse yml from docker compose output.");
return [];
}

Expand Down Expand Up @@ -272,7 +272,7 @@ protected function getDockerComposeConfigYml(): ?string
{
try {
return $this->helpers->process()->withCommand([
'docker-compose',
'docker', 'compose',
'-f', 'docker-compose.yml',
'-f', 'docker-compose.prod.yml',
'--log-level', 'ERROR',
Expand Down
4 changes: 2 additions & 2 deletions app/Commands/DockerBuildCommand.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ public function configure()
*/
public function handle()
{
$requiredBinaries = ['docker', 'docker-compose'];
$requiredBinaries = ['docker'];

if ($this->helpers->checks()->checkAndReportMissingBinaries($this, $requiredBinaries)) {
return 1;
Expand Down Expand Up @@ -96,7 +96,7 @@ protected function callBuild(string $tag, string $service = null): bool
{
try {
$this->helpers->process()->withCommand([
'docker-compose',
'docker', 'compose',
'-f', 'docker-compose.yml',
'-f', 'docker-compose.prod.yml',
'build', '--no-cache', $service
Expand Down
9 changes: 7 additions & 2 deletions app/Commands/DockerPushCommand.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ public function configure()
*/
public function handle()
{
$requiredBinaries = ['docker', 'docker-compose', 'aws'];
$requiredBinaries = ['docker', 'aws'];

if ($this->helpers->checks()->checkAndReportMissingBinaries($this, $requiredBinaries)) {
return 1;
Expand Down Expand Up @@ -96,7 +96,7 @@ protected function callPush(string $tag, string $service = null): bool
{
try {
$this->helpers->process()->withCommand([
'docker-compose',
'docker', 'compose',
'-f', 'docker-compose.yml',
'-f', 'docker-compose.prod.yml',
'push', $service
Expand All @@ -112,4 +112,9 @@ protected function callPush(string $tag, string $service = null): bool

return true;
}

protected function envDockerComposeFileName(string $environment): string
{
return "docker-compose.{$environment}.yml";
}
Comment thread
spamoom marked this conversation as resolved.
Outdated
}
8 changes: 4 additions & 4 deletions composer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@
}
],
"require": {
"php": "^7.3",
"php": "^7.4 | ^8.0",
"laminas/laminas-text": "^2.7",
"laravel-zero/framework": "^7.0",
"nunomaduro/laravel-console-menu": "^3.0",
"laravel-zero/framework": "^8.0",
"nunomaduro/laravel-console-menu": "^3.1",
"padraic/phar-updater": "^1.0.6",
"symfony/yaml": "^5.0"
},
"require-dev": {
"fzaninotto/faker": "^1.9",
"mockery/mockery": "^1.3.1",
"phpunit/phpunit": "^8.5"
"phpunit/phpunit": "^9.3"
},
"autoload": {
"psr-4": {
Expand Down
Loading