Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize deployment setup time #795

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion inventory/environ.py
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,7 @@ def getSplunkBuildAuth(vars_scope):
"""
Load username and password to be used in basic auth when fetching splunk build or apps
"""
vars_scope["splunk"]["artifact_auth_user"] = os.environ.get("ARTIFACTORY_USER", vars_scope["splunk"].get("artifact_auth_user"))
vars_scope["splunk"]["artifact_auth_user"] = os.environ.get("ARTIFACTORY_USER", vars_scope["splunk"].get("basic_auth_user"))
vars_scope["splunk"]["artifact_auth_pass"] = os.environ.get("ARTIFACTORY_TOKEN", vars_scope["splunk"].get("artifact_auth_pass"))

def getSplunkApps(vars_scope):
Expand Down
15 changes: 13 additions & 2 deletions roles/splunk_common/handlers/restart_splunk.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
---
- name: Check if Splunk is running
command: "{{ splunk.exec }} status"
become: yes
become_user: "{{ splunk.user }}"
register: check_result
ignore_errors: yes

- name: "Restart the splunkd service - Via CLI"
command: "{{ splunk.exec }} restart --answer-yes --accept-license"
become: yes
Expand All @@ -7,7 +14,9 @@
until: task_result.rc == 0
retries: "{{ retry_num }}"
delay: "{{ retry_delay }}"
when: not splunk.enable_service
when:
- not splunk.enable_service
- "'splunkd is running' in check_result.stdout"

- name: "Restart the splunkd service - Via Linux systemd or init"
service:
Expand All @@ -26,4 +35,6 @@
- name: "Wait for splunkd management port"
wait_for:
port: "{{ splunk.svc_port }}"
when: splunk.splunk_http_enabled|bool == true
when:
- "'splunkd is running' in check_result.stdout"
- splunk.splunk_http_enabled|bool == true
5 changes: 5 additions & 0 deletions roles/splunk_common/tasks/enable_splunkd_ssl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@
become_user: "{{ splunk.user }}"
when: "'enable' in splunk.ssl"

- name: Set HTTPS as prefix after enabling it
set_fact:
cert_prefix: https
when: "'enable' in splunk.ssl"

- name: Set Splunkd CA
ini_file:
dest: "{{ splunk.home }}/etc/system/local/server.conf"
Expand Down
7 changes: 1 addition & 6 deletions roles/splunk_common/tasks/install_splunk_tgz.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,7 @@
splunk_replaced_build_location: "{{ splunk.build_location | regex_replace('^file://(.*)$', '\\1') }}"

- name: Install Splunk (Linux) from local build location
unarchive:
src: "{% if splunk.build_location is match('^(https?)://.*') %}{{ splunk_downloaded_build_location }}{% else %}{{ splunk_replaced_build_location }}{% endif %}"
dest: "{{ splunk.opt }}"
owner: "{{ splunk.user }}"
group: "{{ splunk.group }}"
remote_src: yes
shell: tar --no-same-owner --no-same-permissions -x --owner={{ splunk.user }} --group={{ splunk.group }} -f {% if splunk.build_location is match('^(https?)://.*') %}{{ splunk_downloaded_build_location }}{% else %}{{ splunk_replaced_build_location }}{% endif %} -C {{ splunk.opt }} && chown -R {{ splunk.user }}:{{ splunk.group }} /opt/splunk/*
register: install_result
until: install_result is succeeded
retries: "{{ retry_num }}"
Expand Down
9 changes: 2 additions & 7 deletions roles/splunk_common/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,14 +122,9 @@
- include_tasks: enable_dsp.yml
when: "'dsp' in splunk and 'enable' in splunk.dsp and splunk.dsp.enable"

- include_tasks: start_splunk.yml

- include_tasks: set_certificate_prefix.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also this playbook?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This playbook, set_certificate_prefix I essentially incorporated into roles/splunk_common/tasks/enable_splunkd_ssl.yml. If we turn on ssl we might as well set the certificate prefix in the same file instead of having two different steps.

when: splunk.splunk_http_enabled|bool == true

- include_tasks: clean_user_seed.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to remove this playbook from the repository? it's not used anywhere else.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don'r recall exactly, but I think the user seed is used to determine if this is the first time Splunk is bootstrapped or not. I remember running into a problem when doing upgrade and when using local dev build because of it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this one I don't have strong preferences about, I couldn't find anything in the docs about what happens if we keep this around.


- include_tasks: add_splunk_license.yml

- include_tasks: disable_popups.yml
when: "'disable_popups' in splunk and splunk.disable_popups | bool"

- include_tasks: start_splunk.yml
20 changes: 8 additions & 12 deletions roles/splunk_common/tasks/set_as_license_slave.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,13 @@
port: "{{ splunk.license_master_url | urlsplit('port') }}"
no_log: "{{ hide_password }}"

- name: Set node as license slave
command: "{{ splunk.exec }} edit licenser-localslave -master_uri {{ splunk.license_master_url }} -auth '{{ splunk.admin_user }}:{{ splunk.password }}'"
- name: Set node as license peer - FAST
ini_file:
dest: "{{ splunk.home }}/etc/system/local/server.conf"
section: license
option: manager_uri
value: "https://lm1:8089"
owner: "{{ splunk.user }}"
group: "{{ splunk.group }}"
become: yes
become_user: "{{ splunk.user }}"
register: linux_set_lic_slave
until: linux_set_lic_slave.rc == 0
changed_when: linux_set_lic_slave.rc == 0 and "licenser-localslave object has been edited" in linux_set_lic_slave.stdout
failed_when: linux_set_lic_slave.rc !=0 and "does not support being a remote master" in linux_set_lic_slave.stderr
retries: "{{ retry_num }}"
delay: "{{ retry_delay }}"
notify:
- Restart the splunkd service
ignore_errors: yes
no_log: "{{ hide_password }}"
13 changes: 0 additions & 13 deletions roles/splunk_search_head/tasks/search_head_clustering.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,16 +75,3 @@
retries: "{{ shc_sync_retry_num }}"
delay: "{{ retry_delay }}"
no_log: "{{ hide_password }}"

- name: Destructive sync search head
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we instead make it optional? would it cause any issues while performing upgrades?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of those tasks that took almost 5 minutes to complete. According to this doc https://docs.splunk.com/Documentation/Splunk/9.2.0/DistSearch/HowconfrepoworksinSHC#Why_a_recovering_member_might_need_to_resync_manually we shouldn't need to perform a destructive sync unless the cluster is recovering a member, and then it's only necessary on that one instance. We always could leave this off by default and have some flag to run it if possible.

command: "{{ splunk.exec }} resync shcluster-replicated-config -auth {{ splunk.admin_user }}:{{ splunk.password }}"
become: yes
become_user: "{{ splunk.user }}"
when: not splunk_search_head_captain | bool
register: task_result
changed_when: task_result.rc == 0
failed_when: task_result.rc !=0 and "this instance is the captain" not in task_result.stderr
until: task_result.rc == 0 or "this instance is the captain" in task_result.stderr
retries: "{{ shc_sync_retry_num }}"
delay: "{{ retry_delay }}"
no_log: "{{ hide_password }}"