Skip to content

Conversation

@PhillypHenning
Copy link
Contributor

@PhillypHenning PhillypHenning commented Jul 20, 2022

Description

Adding the required property attribute to the DeCLI schema parser

Type of change

Please delete options that are not relevant.

  • [X ] New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Tested using the KitchenSink image with 3 ops_repos; Terraform, Ansible, Cloudformation

Logs
#185

@PhillypHenning PhillypHenning added the enhancement ✨ New feature or request label Jul 20, 2022
@PhillypHenning PhillypHenning added this to the BitOps v2.0.0 milestone Jul 20, 2022
@PhillypHenning PhillypHenning self-assigned this Jul 20, 2022
@PhillypHenning PhillypHenning merged commit 928390a into plugins Jul 20, 2022
@PhillypHenning PhillypHenning deleted the plugins-required branch July 20, 2022 17:28
PhillypHenning added a commit that referenced this pull request Aug 3, 2022
* Plugin implementation

* lifecycle hooks and install plugins script

* reordered plugin execution

* pass CLI_OPTIONS to plugin

* Call plugin install.sh on dockerbuild

* Fix plugin install script

* docs

* update docs

* Support empty plugin.config.yml

* rename plugin.config.yml to .yaml

* support local plugins

* added default plutins

* updated plugin urls

* added dockerfile fore core and plugins

* added bitops core and plugins

* updated plugins location

* added azure setup

* added gcp and azure cloud providers

* updated permissions

* added gcp setup script

* updated deploy.py

* fixed helm deployment issues

* build and publish image for testing

* updated build tag

* added deploy script

* updated image name

* debug registry url

* troubleshoot publishing

* updated publishing logic

* updated build tag

* testing cloudformation

* add azure and gcp docs

* gcp terraform example

* azure terraform example

* updated install_tools with other cloud providers

* updated gcp and azure scripts to use proper names

* Initial review and changes

* removing plugins.config.yaml

* Initial review and changes (#149)

* Initial review and changes

* removing plugins.config.yaml

* Consolidated into python; install_plugins.py, deploy_plugins.py & utilities

Signed-off-by: PhillypHenning <phillyp.henning@gmail.com>

* Adding plugins.py "main" file

* removing dead files

* Updating config and schema

* initial pass with branch, tags and versions

* Adding gitpython module + usage

* Adding install|deploy script options

* Consolidating deployment into dpeloy_plugins.py

* consolidating deployment into deploy_plugins

* Update requirements.txt

* Update Dockerfile-plugins

* Separating the cloudprovider and deployment tools in build.config, updated install logic to support

* Adding recursive schema keys loader

* Dead/debug code cleanup

* dead/debug code removal

* Adding Schema_Parser (WIP)

* Adding fast fail bitops mode option

* Updating Dockerfiles

* adding logging

* updating core files

* adding "settings" file

* EoD

* Updating utilities

* deploy plugins functionality completed 75% - deploy script being called, before|after hooks aren't

* updated plugins in build config

* fixed typo

* Clean up

* adding debug log for envir vars

* updating plugins file

* updating deploy script invoke

* Adding logging and try block for running deploy script

* updating path

* updating dockerfile-core

* Adding schema documentation

* Adding ops_repo.source attribute

* adding example to ops_repo.source attribute

* Small updates

* Updating configuration and ENV var settings (X,Y,Z) (ENV VAR, CONFIG, DEFAULT)

* Adding command line arguments

* updating gitignore

* Updating Documentation

* Using settings BITOPS_logging_level

* more logging!

* Updating README with run command

* Round 1 - Updating install logic to harden and expand functionality

* formatting

* Moving floating points and start work on plugins deployments

* Adding schemas

* adding a few resources to BitOps core image

* Adding colors to logging

* General work on the whole project

* install and deploy scripts functioning

* adding `required` property

* Update to the readme

* added local tools to support rapid development

* Update README.md

* Update README.md

* cleanup

* Restructure folders and files for BitOps V2 (#171)

Co-authored-by: Shyam <shyam.rayaprolu@gmail.com>

* updated new bitops config yaml

* reset the config yaml

* fixed install_plugins.py to support bitops.config

* test all plugins installation

* Deployment (#172)

* fixed deploy_plugins.py

* commented deployment sequence

* error handling

* fixed log level

* updated bitops config

* fixed the config name

* adding basic.bitops.config.yaml (documentation)

* cleaning up more files

* Adding Schema and config

* Deployment (#174)

* fixed deploy_plugins.py

* commented deployment sequence

* error handling

* fixed log level

* updated bitops config

* fixed the config name

* updated install_plugins.py as per new changes

* Adding stack-action and cleanup

* adding prebuilt-config, cleanup (#175)

* Ops repo deployment using BitOps V2 (#176)

* fixed deploy plugins

* updated the ops repo folder name

* updating deploy.py

* Adding files and folders breakdown (for devs)

* Update deploy_plugins.py

* Updating exported ENV vars

* Small update to util

* Adding aws+terraform example

* plugin.config.yaml related changes (#178)

* Feature to invoke core bitops utilities from plugins (#180)

* plugin.config.yaml related changes

* added steps to invoke util functions

* fixed the "mode" for parsing

* added boto3

* glibc compatibility

* Before and after hooks added

* updating example doc

* Separated before+after hooks, added dependency awareness

* Adding documentation kv lookup and lookup function

* Adding error catch

* added all plugins to config

* updating config aws+terraform

* Adding github extras

* updating tags to `plugins`

* testing

* rapid testing

* removing ops_repo from prebuilt-config

* updating Dockerfiles and bitops.configs

* Adding image tag

* updating tag names

* updating publish.sh with the "latest" = "KitchenSink"

* Updating publish.sh

* Adding AwsAnsible and AwsTerraformHelm images

* taking out dockerfile copy

* adding image prefix

* updating publish.sh

* rapid testing

* updating workflow names and adding core build

* updating prefix

* Updating image_tag with prefix

* Updating prebuilt-config Dockerfiles

* Updating latest tag handle

* Using Dockerfile template

* Update core build on paths

* Updating the paths for core

* formatting

* Adding additional ignores

* Updating bitops-tag value

* create v2.0.0 tag

* Update deploy_plugins.py

* Update build-and-publish-core.yml

* removed awscliv2 installation from dockerfile

* Adding workflow dispatch

* Update bitops-tag

* Updating log level to warn (from error)

* Adding folder check (no hooks)

* dirty bump

* Update build-and-publish-core.yml

* Adding required property (#204)

* Creating 2.0.0-base image (#212)

* Creating 2.0.0-base image

* updating folder names and ci pipelines

* Update bitops-tag

* Plugins ci (#213)

* Creating 2.0.0-base image

* updating folder names and ci pipelines

* adding the kitchensink

* Plugins ci (#214)

* Creating 2.0.0-base image

* updating folder names and ci pipelines

* adding the kitchensink

* adding IMAGE_POSTFIX

* Update build-and-publish-prebuilt.yml

* Adding stderr to failed logging

* Plugins doc (#225)

* Updating README

* capturing cleanup here

* Updating the splash text

* adding example-plugin

* 3 more doc page changes

* Adding two new pages for development

* Adding 2.0.0 migration docs

* Feedback

* centering image

* wording

* Fix plugins base image requiring additional files ONBUILD

Make sure only `bitops.config.yaml` is a required file to build the plugins Docker image based on the base image.

Fixes
```
 => ERROR [3/1] COPY scripts/ ./scripts                                                                                               0.0s
 => CACHED [4/1] COPY bitops.config.yaml .                                                                                            0.0s
 => ERROR [5/1] COPY bitops.schema.yaml .
```

* Updating base image name to `plugins-base` (#234)

* Updating base image name to `plugins-base`

* Update build-and-publish-base.yml

* Updating from review

* adding release trigger

* updating doc build trigger

* Plugins doc (#229)

* Updating README

* capturing cleanup here

* Updating the splash text

* adding example-plugin

* 3 more doc page changes

* Adding two new pages for development

* Adding 2.0.0 migration docs

* Feedback

* Update README.md

* feedback

* feedback

* Updating kitchensink to omnibus

* adding CI doc

* use yaml instead of yml

* use BitOps instead of bitops

* small tweaks to the main readme

* verbiage updates to plugins.md

* some additional docs tweaks

* adding glossary.md

* Update glossary.md

* Update glossary.md

Co-authored-by: Mick McGrath <mick@bitovi.com>

* updating CI pipelines

* updating CI

* updating CI

* updating conditionals

* updating conditionals

* updating supporting files for plugins

* Updating CI pipelines

* updating bitops_tag

* updating bitops_tag

* echo sources..

* updating base dockerfile (removing wildcard)

* updating tag

* updating triggers

* updating github actions to use yaml

* Updating IMAGE_TAG for release and workflows_dis

* adding required keywork for base_tag

* Fix Docker image tag for plugins-base

* updating install to ONBUILD

* Updating prebuilt CI

* updating prebuilt CI

* Updating prebuilt CI

* Updating prebuilt CI

* rapid testing - prebuilt CI

* rapid testing - prebuilt CI

* Updating prebuilt CI

* Updating doc - CI

* Updating prebuilt CI

* adding `plugins-tag`

* Updating workflow disbatch

* Updating doc

* add local docs guide and docker-compose file

* update docker creds for all pipelines to use org-hosted creds

* Remove scripts and bitops.schema.yaml from the Dockerfile ONBUILD requirements

The plugins-base ONBUILD stage should require `bitops.config.yaml` file only per plugins documentation so users can build their own BitOps image

* Fix broken bullet lists in the plugins docs

* Remove outdated publish-docs.yml for master branch

It's replaced by the publish-docs.yaml for the `main` branch

* Adding doc notice to wiki pages

* Add error reporting if bitops.config.yaml doesn't have 'bitops' at a root level (#236)

* Use 'bitops:base' in the documentation for plugins build

* Additional docker tagging for Omnibus and Base (#244)

* Fix 'disbatch' -> 'dispatch' typo in the CD workflows

* Additional image tagging for ':latest', ':dev', and ':base'

* Additional image tagging for ':latest', ':dev', and ':base'

* Adjust the base workflow with additional release tag

* Remove the 'v' prefix from the v1.2.3 for docker tagging

* Match image tag that ends with 'omnibus'

* Fix plugins install not returning correct exit code on failure (#242)

* Fix plugins install not returning correct exit code on failure

Fixes #239

* Use the proper logger.error on a plugin failure

Co-authored-by: Ian Hill <ian@bitovi.com>
Co-authored-by: PhillypHenning <phillyp.henning@gmail.com>
Co-authored-by: Shyam <shyam.rayaprolu@gmail.com>
Co-authored-by: PhillypHenning <phil8t8@gmail.com>
Co-authored-by: Eugen C <1533818+armab@users.noreply.github.com>
Co-authored-by: Mick McGrath <mick@bitovi.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement ✨ New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants