Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add resque restart workers task #284

Merged
merged 6 commits into from
Aug 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 26 additions & 32 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This configuration was generated by
# `rubocop --auto-gen-config --auto-gen-only-exclude --exclude-limit 10000`
# on 2022-10-18 21:05:49 UTC using RuboCop version 1.36.0.
# on 2024-07-18 19:32:08 UTC using RuboCop version 1.36.0.
# The point is for the user to remove these configuration records
# one by one as the offenses are removed from the code base.
# Note that changes in the inspected code, or installation of new
Expand Down Expand Up @@ -28,7 +28,7 @@ Layout/ElseAlignment:
Exclude:
- 'config/initializers/patch_failure_app.rb'

# Offense count: 49
# Offense count: 50
# This cop supports safe autocorrection (--autocorrect).
Layout/EmptyLineAfterGuardClause:
Exclude:
Expand Down Expand Up @@ -148,7 +148,7 @@ Layout/ExtraSpacing:
- 'config/deploy.rb'
- 'config/routes.rb'

# Offense count: 93
# Offense count: 94
# This cop supports safe autocorrection (--autocorrect).
# Configuration parameters: AllowMultipleStyles, EnforcedHashRocketStyle, EnforcedColonStyle, EnforcedLastArgumentHashStyle.
# SupportedHashRocketStyles: key, separator, table
Expand Down Expand Up @@ -326,12 +326,6 @@ Lint/MixedRegexpCaptureTypes:
Exclude:
- 'app/models/deposit.rb'

# Offense count: 1
# This cop supports unsafe autocorrection (--autocorrect-all).
Lint/NonDeterministicRequireOrder:
Exclude:
- 'spec/rails_helper.rb'

# Offense count: 4
# This cop supports safe autocorrection (--autocorrect).
Lint/RedundantCopDisableDirective:
Expand Down Expand Up @@ -378,7 +372,7 @@ Lint/UselessAssignment:
- 'app/models/statistic.rb'
- 'lib/tasks/load.rake'

# Offense count: 43
# Offense count: 44
# Configuration parameters: AllowedMethods, AllowedPatterns, IgnoredMethods, CountRepeatedAttributes, Max.
Metrics/AbcSize:
Exclude:
Expand All @@ -395,7 +389,6 @@ Metrics/AbcSize:
- 'app/forms/email_author_reports_form.rb'
- 'app/forms/request_agreements_form.rb'
- 'app/forms/usage_statistics_reports_form.rb'
- 'app/helpers/application_helper.rb'
- 'app/helpers/catalog_helper.rb'
- 'app/helpers/homepage_helper.rb'
- 'app/helpers/use_and_reproduction_helper.rb'
Expand All @@ -414,6 +407,7 @@ Metrics/AbcSize:
- 'lib/academic_commons/indexer.rb'
- 'lib/academic_commons/metrics/author_affiliation_report.rb'
- 'lib/academic_commons/metrics/output.rb'
- 'lib/tasks/resque.rake'

# Offense count: 12
# Configuration parameters: CountComments, Max, CountAsOne, ExcludedMethods, AllowedMethods, AllowedPatterns, IgnoredMethods, inherit_mode.
Expand All @@ -438,7 +432,7 @@ Metrics/ClassLength:
- 'app/models/solr_document.rb'
- 'lib/academic_commons/metrics/usage_statistics.rb'

# Offense count: 9
# Offense count: 10
# Configuration parameters: AllowedMethods, AllowedPatterns, IgnoredMethods, Max.
Metrics/CyclomaticComplexity:
Exclude:
Expand All @@ -451,8 +445,9 @@ Metrics/CyclomaticComplexity:
- 'app/models/statistic.rb'
- 'app/models/user.rb'
- 'lib/academic_commons/metrics/author_affiliation_report.rb'
- 'lib/tasks/resque.rake'

# Offense count: 42
# Offense count: 44
# Configuration parameters: CountComments, Max, CountAsOne, ExcludedMethods, AllowedMethods, AllowedPatterns, IgnoredMethods.
Metrics/MethodLength:
Exclude:
Expand Down Expand Up @@ -489,8 +484,9 @@ Metrics/MethodLength:
- 'lib/academic_commons/metrics/author_affiliation_report.rb'
- 'lib/academic_commons/metrics/output.rb'
- 'lib/academic_commons/metrics/usage_statistics.rb'
- 'lib/tasks/resque.rake'

# Offense count: 9
# Offense count: 10
# Configuration parameters: AllowedMethods, AllowedPatterns, IgnoredMethods, Max.
Metrics/PerceivedComplexity:
Exclude:
Expand All @@ -503,6 +499,7 @@ Metrics/PerceivedComplexity:
- 'app/models/statistic.rb'
- 'app/models/user.rb'
- 'lib/academic_commons/metrics/author_affiliation_report.rb'
- 'lib/tasks/resque.rake'

# Offense count: 10
# Configuration parameters: EnforcedStyleForLeadingUnderscores.
Expand Down Expand Up @@ -581,7 +578,7 @@ RSpec/AnyInstance:
- 'spec/controllers/users/omniauth_callbacks_controller_spec.rb'
- 'spec/support/ldap_mock.rb'

# Offense count: 15
# Offense count: 16
RSpec/Capybara/VisibilityMatcher:
Exclude:
- 'spec/features/item_page_spec.rb'
Expand All @@ -594,7 +591,7 @@ RSpec/ContextMethod:
- 'spec/models/email_preference_spec.rb'
- 'spec/models/user_spec.rb'

# Offense count: 98
# Offense count: 99
# Configuration parameters: Prefixes, AllowedPatterns.
# Prefixes: when, with, without
RSpec/ContextWording:
Expand Down Expand Up @@ -676,7 +673,7 @@ RSpec/EmptyLineAfterExampleGroup:
- 'spec/helpers/use_and_reproduction_display_helper_spec.rb'
- 'spec/models/featured_search_spec.rb'

# Offense count: 43
# Offense count: 45
# This cop supports safe autocorrection (--autocorrect).
RSpec/EmptyLineAfterFinalLet:
Exclude:
Expand Down Expand Up @@ -811,7 +808,7 @@ RSpec/MultipleExpectations:
- 'spec/models/statistic_spec.rb'
- 'spec/support/authorization_for_controller_tests.rb'

# Offense count: 17
# Offense count: 18
# Configuration parameters: IgnoreSharedExamples.
RSpec/NamedSubject:
Exclude:
Expand All @@ -820,7 +817,7 @@ RSpec/NamedSubject:
- 'spec/academic_commons/metrics/usage_statistics_spec.rb'
- 'spec/controllers/solr_documents_controller_spec.rb'

# Offense count: 13
# Offense count: 14
# Configuration parameters: Max, AllowedGroups.
RSpec/NestedGroups:
Exclude:
Expand Down Expand Up @@ -886,7 +883,7 @@ Rails/DynamicFindBy:
- 'app/models/user.rb'
- 'lib/academic_commons/metrics/author_affiliation_report.rb'

# Offense count: 16
# Offense count: 19
# Configuration parameters: EnforcedStyle.
# SupportedStyles: slashes, arguments
Rails/FilePath:
Expand All @@ -899,6 +896,7 @@ Rails/FilePath:
- 'lib/academic_commons/indexer.rb'
- 'lib/tasks/ac/bots.rake'
- 'lib/tasks/load.rake'
- 'lib/tasks/resque.rake'
- 'spec/factories/deposit.rb'
- 'spec/support/test_log.rb'
- 'spec/tasks/sitemap_spec.rb'
Expand Down Expand Up @@ -968,7 +966,7 @@ Rails/Present:
Exclude:
- 'app/models/notification.rb'

# Offense count: 2
# Offense count: 1
# This cop supports unsafe autocorrection (--autocorrect-all).
# Configuration parameters: Include.
# Include: **/Rakefile, **/*.rake
Expand Down Expand Up @@ -1008,19 +1006,13 @@ Security/Open:
- 'lib/tasks/ac/index.rake'
- 'lib/tasks/ac/reindex.rake'

# Offense count: 1
# This cop supports unsafe autocorrection (--autocorrect-all).
Security/YAMLLoad:
Exclude:
- 'config/initializers/resque.rb'

# Offense count: 1
# This cop supports safe autocorrection (--autocorrect).
Style/BlockComments:
Exclude:
- 'spec/spec_helper.rb'

# Offense count: 1
# Offense count: 2
# This cop supports safe autocorrection (--autocorrect).
# Configuration parameters: EnforcedStyle, ProceduralMethods, FunctionalMethods, AllowedMethods, AllowedPatterns, IgnoredMethods, AllowBracesOnProceduralOneLiners, BracesRequiredMethods.
# SupportedStyles: line_count_based, semantic, braces_for_chaining, always_braces
Expand All @@ -1030,6 +1022,7 @@ Style/BlockComments:
Style/BlockDelimiters:
Exclude:
- 'app/presenters/usage_statistics_presenter.rb'
- 'lib/tasks/resque.rake'

# Offense count: 1
# This cop supports unsafe autocorrection (--autocorrect-all).
Expand Down Expand Up @@ -1517,7 +1510,7 @@ Style/StringConcatenation:
- 'lib/hyacinth.rb'
- 'lib/tasks/load.rake'

# Offense count: 106
# Offense count: 108
# This cop supports safe autocorrection (--autocorrect).
# Configuration parameters: EnforcedStyle, ConsistentQuotesInMultiline.
# SupportedStyles: single_quotes, double_quotes
Expand Down Expand Up @@ -1561,14 +1554,14 @@ Style/StringLiteralsInInterpolation:
- 'app/helpers/catalog_helper.rb'
- 'lib/academic_commons/indexer.rb'

# Offense count: 13
# Offense count: 16
# This cop supports unsafe autocorrection (--autocorrect-all).
# Configuration parameters: AllowMethodsWithArguments, AllowedMethods, AllowedPatterns, IgnoredMethods, AllowComments.
# AllowedMethods: respond_to, define_method
Style/SymbolProc:
Exclude:
- 'app/api/v1/entities/short_record.rb'
- 'app/api/v1/entities/full_record.rb'
- 'app/api/v1/entities/short_record.rb'
- 'app/models/statistic.rb'

# Offense count: 1
Expand Down Expand Up @@ -1610,7 +1603,7 @@ Style/WhileUntilDo:
- 'lib/academic_commons/metrics/author_affiliation_report.rb'
- 'lib/tasks/ac.rake'

# Offense count: 164
# Offense count: 174
# This cop supports safe autocorrection (--autocorrect).
# Configuration parameters: Max, AllowHeredoc, AllowURI, URISchemes, IgnoreCopDirectives, AllowedPatterns, IgnoredPatterns.
# URISchemes: http, https
Expand Down Expand Up @@ -1647,6 +1640,7 @@ Layout/LineLength:
- 'lib/academic_commons/indexer.rb'
- 'lib/academic_commons/metrics/usage_statistics.rb'
- 'lib/hyacinth.rb'
- 'lib/tasks/resque.rake'
- 'spec/academic_commons/featured_searches_spec.rb'
- 'spec/academic_commons/metrics/author_affiliation_report_spec.rb'
- 'spec/academic_commons/metrics/usage_statistics_spec.rb'
Expand Down
6 changes: 3 additions & 3 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ gem 'nokogiri', '~> 1.15.2'
gem 'okcomputer'
gem 'premailer-rails'
gem 'rainbow'
gem 'resque', '~> 1.27'
gem 'resque', '~> 2.0'
gem 'rinku'
gem 'rsolr-ext'
gem 'rubyzip', require: 'zip'
Expand All @@ -49,8 +49,8 @@ gem 'will_paginate'
gem 'wowza-secure_token'

# Database
gem 'mysql2'
gem 'sqlite3', '>= 1.3.5'
gem 'mysql2', '>= 0.5.6'
gem 'sqlite3', '~> 1.4'

group :development do
gem 'listen'
Expand Down
29 changes: 14 additions & 15 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,7 @@ GEM
ruby2_keywords (~> 0.0.1)
mustermann-grape (1.0.2)
mustermann (>= 1.0.0)
mysql2 (0.5.4)
mysql2 (0.5.6)
net-imap (0.4.0)
date
net-protocol
Expand Down Expand Up @@ -391,7 +391,7 @@ GEM
rack (2.2.4)
rack-accept (0.4.5)
rack (>= 0.4)
rack-protection (2.2.0)
rack-protection (2.2.1)
rack
rack-proxy (0.7.2)
rack
Expand Down Expand Up @@ -448,18 +448,17 @@ GEM
rdf-xsd (1.99.0)
rdf (~> 1.99)
redis (4.6.0)
redis-namespace (1.8.2)
redis (>= 3.0.4)
redis-namespace (1.11.0)
redis (>= 4)
regexp_parser (2.6.0)
responders (3.0.1)
actionpack (>= 5.0)
railties (>= 5.0)
resque (1.27.4)
resque (2.6.0)
mono_logger (~> 1.0)
multi_json (~> 1.0)
redis-namespace (~> 1.3)
redis-namespace (~> 1.6)
sinatra (>= 0.9.2)
vegas (~> 0.1.2)
resque-scheduler (4.10.0)
mono_logger (~> 1.0)
redis (>= 3.3)
Expand Down Expand Up @@ -557,10 +556,10 @@ GEM
simplecov_json_formatter (~> 0.1)
simplecov-html (0.12.3)
simplecov_json_formatter (0.1.4)
sinatra (2.2.0)
sinatra (2.2.1)
mustermann (~> 1.0)
rack (~> 2.2)
rack-protection (= 2.2.0)
rack-protection (= 2.2.1)
tilt (~> 2.0)
sitemap_generator (6.2.1)
builder (~> 3.0)
Expand All @@ -583,7 +582,9 @@ GEM
actionpack (>= 5.2)
activesupport (>= 5.2)
sprockets (>= 3.0.0)
sqlite3 (1.4.2)
sqlite3 (1.7.3)
mini_portile2 (~> 2.8.0)
sqlite3 (1.7.3-x86_64-darwin)
sshkit (1.21.2)
net-scp (>= 1.1.2)
net-ssh (>= 2.8.0)
Expand All @@ -606,8 +607,6 @@ GEM
unf_ext (0.0.8.2)
unicode (0.4.4.4)
unicode-display_width (2.3.0)
vegas (0.1.11)
rack (>= 1.0.0)
view_component (2.71.0)
activesupport (>= 5.0.0, < 8.0)
concurrent-ruby (~> 1.0)
Expand Down Expand Up @@ -680,7 +679,7 @@ DEPENDENCIES
listen
mail (~> 2.8)
multipart-post (~> 2.0.0)
mysql2
mysql2 (>= 0.5.6)
net-scp (~> 4.0.0)
net-ssh (~> 7.2.0)
nokogiri (~> 1.15.2)
Expand All @@ -689,7 +688,7 @@ DEPENDENCIES
puma (~> 5.2)
rails (= 6.0.4)
rainbow
resque (~> 1.27)
resque (~> 2.0)
rinku
rsolr-ext
rspec-its
Expand All @@ -701,7 +700,7 @@ DEPENDENCIES
sitemap_generator
solr_wrapper (~> 4.0)
spring
sqlite3 (>= 1.3.5)
sqlite3 (~> 1.4)
turbolinks
uglifier
unicode
Expand Down
2 changes: 1 addition & 1 deletion app/jobs/sword_deposit_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ def perform(deposit)

# Send request to SWORD
begin
response = HTTP.timeout(write: 60, connect: 60, read: 60)
response = HTTP.timeout(write: 60, connect: 60, read: 90)
.basic_auth(user: credentials['user'], pass: credentials['password'])
.headers(content_type: 'application/zip')
.post(credentials['url'], body: deposit.sword_zip)
Expand Down
2 changes: 1 addition & 1 deletion config/deploy.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
lock '3.17.0'
lock '3.17.3'

set :instance, 'ac'
set :application, 'academiccommons'
Expand Down
3 changes: 2 additions & 1 deletion config/deploy/prod.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
role :resque_worker, "ac-rails-#{fetch(:stage)}1.cul.columbia.edu"
role :resque_scheduler, "ac-rails-#{fetch(:stage)}1.cul.columbia.edu"

set :workers, { '*' => 6 }
set :workers,
YAML.load_file(File.expand_path('../resque.yml', __dir__), aliases: true)[fetch(:stage).to_s]['workers']

# We default to storing PID files in a tmp/pids folder in your shared path.
# set :resque_pid_path, -> { File.join(shared_path, 'tmp', 'pids') }
Expand Down
Loading
Loading