SPARCRequest Wiki

SPARCRequest Version 3.3.0 & SPARCFulfillment Version 2.8.0

SPARCRequest Version 3.3.0 New Features


1.(SPARC Gems) Updates to Gems
In this release, third party gems have been updated to the latest version, such as factory_bot_rails (from v4.10.0 to 4.11.0), rspec-rails (3.7.2 to 3.8.0), and newrelic_rpm (5.2.0.345 to 5.3.0.346).

2.(SPARC Codes) GitHub Dependabot Integration
In previous releases (since 2017), the Hakiri Facets security scan has been used to check on any necessary gemfile updates. These updates were performed to avoid potential system vulnerabilities. To improve this process, Dependabot has been integrated into the SPARCRequest code base, which checks the gem updates with dependency and security issues and generates a pull request automatically to the repository when an gem update is suggested.3. (SPARC Codes) Upgrade Rails to v5.2.0

In this release, the Rails version has been updated to v5.2.0 for maintenance and better performance.

4. (SPARC Codes) Remove Service and Service Provider Relation in Models

The relation between services and service providers was removed in a previous migration, but there were still leftover codes with the relation referenced in some models. The related codes have since been removed.

5. (SPARC Codes) Customizable Time Zone (Utah Contribution)

Previously, SPARCRequest defaulted to Eastern Time Zone and there was no way to change it. The time zone setting has been added into the `.env` file, which allows other institutions to make customizations.

6. (SPARC Codes) Devise Secret Key Security Improvement (Utah Contribution)

Previously, the devise SECRET_KEY was stored in `config/initializers/devise.rb` which made it public. This key is now kept secret, as it is used to validate the authenticity of requests to the server from Devise controllers. This change was made to protect against sending potentially malicious external requests to Devise controllers, which could have lead to compromised user data. In this release, the devise SECRET_KEY has been moved to `.env` and stored locally.

7. (SPARC Codes) Bug Fix for Method Sequence Error in Directory.rb
Refactored methods in app/lib/directory.rb to use the Settings table were causing an issue when trying to run the suite-updater (to refresh SPARC data on local machine). This would have also been an issue with anyone who tried to run a rake task before database existence. This bug has been fixed.

8. (SPARC Codes & Database) Incomplete Status Added to Permissible_values (LA CaTS Contribution)
‘Incomplete’ status has been added to the permissible_values table in this production. The corresponding Available Status and Editable Status records for the incomplete status have been added in the database for each existing organization. The 'Incomplete' status was designed to be utilized for scenarios where information was missing for a request.

9. (SPARC Codes & Database)  Is_available Field and Function Added to Permissible Values
To better manage content of the permissible values list, an "is_available" field has been added to permissible_values table in the release.This was added so each institution could maintain their active content for each category, while keeping the historical record for their inactive content. For example, at MUSC, one of the affiliation types (INBRE) is no longer active. Some of the values on the permissible_values list will need annual updating (such as affiliation_type, when a COBRE or core becomes inactive, or a new COBRE has been established).

10. (SPARC Data) Past Status Data Clean-up
A script was created to clean up the historical data in the past_statuses table. The clean-up was necessary since there were historical records that had missing underscores. For example, "on_hold" was showing as "on hold", "in _process" was showing as "in process," etc.

11. (SPARCRequest) Habitat Build 

A habitat package has been created based on the SPARC release-3.1.0 tag version of the code.  This package has been published to habitat under a new SPARC origin that has been installed and run by MUSC.

12. (SPARCRequest, SPARCDashboard, and SPARC Database) Admin Rate Label Change and Visual Cue
Previously the admin rate was an attribute on the line item level. When an admin user set an admin rate on a service inside of a request, and proceeded to add the same service in SPARCFulfillment (which was not pushed from SPARCRequest), SPARCFulfillment considered it as a service with the normal rate rather than an admin rate. This has since been fixed.

13. (SPARCRequest) Homepage Calendar Bugs
A previous bug was causing loading/refreshing issues on the google calendar section on the SPARCRequest homepage. Recurring events were also not displaying in this section. This bug has since been fixed so that recurring events will now display.

14. (SPARCRequest) Short Interaction (LA CaTS Contribution)

A short interaction form was created for Service Providers to capture "short interactions" with investigators in SPARC. The short interaction (required) data fields include the following:

Subject of Interaction (Dropdown); Type of Interaction (Dropdown); Duration in minutes (text); Investigator Name (text); Investigator Email (text); Investigator Institution (Drowdown); Conversation Note (text)

The short interaction feature has also been made configurable for OS partners.

15. (SPARCRequest) Short Interaction Migrate Institution out of Constants.yml
The institutions used in the "Short Interaction" function were previously being pulled from constants.yml. This has since been migrated to use the institutions from professional_organizations in the database instead and the old attributes have been removed from constants.yml.

16. (SPARCRequest) Step 2: Styling for Finished SSR's
On the SPARCRequest Step 2 (Services and Protocol Information) page, the services contained in an already completed (or other finished_statuses) were previously shown on the list in the center of the page without differentiation from the new/active services. The same styling (pink background highlight, and little lock) that is used for requests in finished statuses (as shown in the shopping cart "Completed" tab) has now been applied to the services listed in the center of this page for consistency.
17. (SPARCRequest Step 2 & SPARCDashboard) "Proxy Rights" Label Consistency
In this production, the label on the "Edit/Add Authorized User" popup window to "Proxy Rights" has been changed to be consistent throughout the system.

18. (SPARCRequest Step 3 and SPARCDashboard) Milestone Language
After receiving feedback from investigators in regard to submission barriers, service start and end dates were not always known, and therefore resulted in a potential hang up on this step. In this production, the language has been changed to incorporate the word "Estimated" in front of Start Date and End Date to clear up any confusion.

19. (SPARCRequest & SPARCDashboard) Timeline Budget Negotiation Metric Fields Missing Cent Digit
Cents were previously displaying correctly in the database but not in the front-end. This has now been fixed for consistency to allow typing down to the cents. Dollar amounts on the protocol level are also now stored as value x 100, to be consistent with all other dollar ($) fields in the system. 

20. (SPARCRequest) Line Item Subject Count Bug: 0
A remaining bug has been fixed from a previous a bug story in v3.2.0, where the Subject Count "N" was not updating to a different value for clinical services. The scenario for line items when N=0 was still not fixed upon submission. 


21. (SPARCDashboard) Users Can't Remove Outdated Calendar Bug
Previously, when the last clinical service was removed from a protocol by a study team user or admin user, the calendar-related fields no longer displayed, however, the calendar was lingering in the back-end. This was causing an error message from the associated clinical services data when there was no clinical service(s) left on this protocol. This bug has been fixed and now automatically removes the calendar-related information (arms, visits, visit_groups, etc) when the last clinical service is removed from a protocol (previous error message below).

22. (SPARCRequest & SPARCDashboard)  Document Types Limitation Upload Bug
In expansion of a previous story from v3.2.0 (#27) where file type parameters were set, users were unable to upload documents with capital letter file types (i.e. .PDF vs .pdf), but were then allowed when converting. All previously defined permitted file types (pdf, doc, docx, rtf, txt, xls, xlsx, csv, ppt, pptx, msg, eml (for emails), jpeg, gif, png, tiff) now also account for capital letter extensions in this production.

23. (SPARCRequest) New protocol Service Request Bug (LA CaTS Contribution)
A Rails 5.2 upgrade bug has been fixed in this release. Authorized users were able to add a document on Step 5: Documents, Notes, & Forms, but there was not a way to assign access. When proceeding to SPARCDashboard the request column was blank and service requests were also missing when clicking inside of a protocol, although they existed in the database. This bug occurred when creating a new protocol.

24. (SPARCRequest & SPARCDashboard) Time stamping (LA CaTS Contribution)

A time stamp has been added to the "Uploaded" column in the documents table on SPARCRequest Step 5, SPARCDashboard documents section, as well as on the SPARCFunding download document page. The uploaded date was only displayed before, but now the time is displayed as well.
25. (SPARCRequest & SPARCDashboard) 
User Update Email Bug
When making user changes on a protocol that have previously been pushed to Epic, emails were also being sent to the users with "Member Only" rights. This has since been fixed and those Authorized Users with "Member Only" proxy rights no longer receive SPARC system email notifications.

26. (SPARCDashboard) Epic Queue Notes
A notes column has been added inside the SPARCDashboard Epic Queue, similar to the service level notes found on the SPARCRequest Step 4: Service Calendar. These notes can only be added once a record has been pushed to Epic and were added to document scenarios that did not fall into normal workflow, or simply needed further clarification. An export function has also been added to the Epic queue notes if users choose to export the notes.

27. (SPARCDashboard & SPARCRequest) Save as Draft Bug When Editing a Request

A previous bug existed where all service requests on an existing protocol were being sent back to "Draft" status. This occurred when using the "Edit" button(s) on SPARCDashboard, adding a service and proceeding to save as draft for an individual request. This has now been fixed to only update the status of the service request to draft and all other service requests remain unchanged.

28. (SPARCDashboard) Coverage Analysis Report Fix to Only Show Hospital Services
Previously the "Coverage Analysis Report" generated from SPARCDashboard was showing every service that existed on a given protocol (with the option of display Chosen Services or All Services). To better improve the Coverage Analysis process, this report now only includes hospital services, which are pertinent to the process. 

The following Epic configurations were made:

  1. If the application has Epic configuration turned on (use_epic = true), on the generated CA report, only services that are set to be pushed to Epic will display (services.send_to_epic = 1).

  2. If the application has Epic configuration turned off (i.e. for institutions not using the SPARC/Epic interface), the current Coverage Analysis report logic is applied where all service are listed (no matter if sending to Epic or not). This allows institutions who are not using the Epic interface to still review protocols using this report, and base their workflow on reviewing CPT-coded service or all services.

29. (SPARCDashboard) Coverage Analysis Report Bug
Previously when running the Coverage Analysis report for a Protocol which contained a large sum of visits (i.e. 100+ visits), the application would error out. This bug has since been fixed to allow for larger protocols with large visit counts.

30. (SPARCDashboard) Export Consolidated Corporate Study Budget Report Fixes
The Consolidated Corporate Study Budget report had several bugs fixed, as well as calculation features added in this production. These fixes and features have been made to assist workflow and for better budget negotiation.


31. (SPARCDashboard Admin Edit) Go to Fulfillment Direct Link
Previously, after a request had been pushed to SPARCFulfillment and the user clicked the "Go to Fulfillment" button, a new tab opened showing the SPARCFulfillment homepage. For better efficiency, this has been changed to link directly to the corresponding request/protocol in SPARCFulfillment.


32. (SPARCDashboard Admin Edit) Status Sorting Bug
Previously, the "status"category of the permissible values were not following the defined sort order, although other fields such as document types and impact area were following the order. This bug has since been fixed.

33. (SPARCDashboard Admin Edit)  Export to Excel Bug
Previously, when a large sum of visits (i.e. 100+ visits) were exported to excel, the function was breaking. This bug was also affecting system-generated emails with large visits counts, resulting in a failure to generate the excel attachment and winding up in the delayed_job queue. This bug has since been fixed.

34. (SPARCDashboard Admin Edit) Send to Epic Button Bug
Previously in SPARCDashboard, the "Send to Epic" button was not showing up for admin users. This bug occurred when a service that was chosen to be as "send to Epic" in SPARCCatalog didn't belong directly to a split/notify organization, but belonged to a child organization of the split/notify organization instead. This bug has since been fixed (previous screenshot below).


35. (SPARCDashboard Admin Edit) Add Non-clinical Services and Edit Quantity Bugs
Previously, a bug was not allowing Admins to 'Add Non-clinical services' in Admin Edit. This has since been fixed and Admins can create and destroy Non-clinical services. Secondly, a bug has been fixed that was not allowing a decimal after a number (i.e. 1.0). This bug has been fixed for system consistency.


36. (SPARCDashboard Admin Edit) Admin Rate Bug

A previous bug has been fixed that did not allow the admin rate to update according to the entered value, although the value was saved in the database multiple times. 



37. (SPARCDashboard Admin Edit) Status Change Bug (LA CaTS Contribution)

A previous bug has been fixed where the status change for a request was not being captured. Instead of creating a new status history the status instead changed to the previous status history (previous bug screenshots below).

38. (SPARCDashboard Admin Edit) Non-Clinical Service Should be Able to Update Unit Type
Previously, only study team members were able to enter the "Unit Requested" for non-clinical services on the SPARCRequest Step 4 page. The "Unit Requested" was also missing on the SPARCDashboard Admin Edit section when editing that activity. In this production, this has been fixed and made consistent with SPARCRequest so that "Unit Requested" is a required field when "Edit Activity" on the services using that field (previous screenshot below).

39. (SPARC_Epic Interface) Billing/Business Manager Logic

Previously, the "Billing/Business Manager" user role in SPARC was mapped to "I RSH 120" role_code in Epic. This has since been changed to "I RSH 110" in Epic instead since this group of users have Epic access/training certification and need the ability to review charges (see SPARC_Epic Interface Guide).


40. (SPARCCatalog & SPARCDashboard) Survey Builder Display Order

In this production, the survey builder order is now defaulting to sort by Access code and version. Previously the order was not displaying correctly.

41. (SPARCCatalog) Form Builder: Validation 

In this production, validations were implemented for users who have access to the form builder (Form Functionality and Survey). The following validations were made so modifications on a form with existing responses won't cause broken data:

  1. Users are not able to edit or delete a currently active form (only inactive forms can be edited).

  2. Users are not able to delete a form that already have submitted responses.

  3. Users can only add non-required new question(s) to an existing form which already has responses. (Adding required question(s) would cause validation issues for existing submissions, as well as potential user confusion about previous submissions).

  4. Users can remove question(s) for an existing form with responses, but they are warned if there are dependency question(s) being removed.

  5. If the form already has responses, changing the type of the question  is not allowed on the form.

  6. Users are not be able to "activate" a survey/form that has no content added. 

42. (SPARCCatalog) Ability to Create New Organizations Without Rights
A bug was found that allowed users without catalog manager rights to create organizations. This has since been fixed and validations are now in place. Users without appropriate access will now see an error message.

43. (SPARCCatalog) Navigation Bar, Site Footer and Logos
The Navbar in SPARCCatalog is now consistent with the Navbar in all other SPARC modules, and the module you are actively in is highlighted. The footer section is also now a working section, consistent with the other SPARC modules.

44. (SPARCCatalog) Bootstrap Catalog

In this production, a major revamp has occurred by bootstrapping SPARCCatalog for better usability, speed, validation, and consistency with the other SPARC modules. Refer to the new SPARCCatalog User Guide for all of the new functionality/fixes.

45. (SPARCCatalog) Show All/Show Available Function Improvement

Visual cues have been improved to better differentiate between active and inactive services and organizations when the user sets the toggle button to "Show All" and "Show Available."

46. (SPARCCatalog) Search Result Display Improvement & Breadcrumb
New functionality has been added to the Search bar in SPARCCatalog to display organizations and services based on the search term, a visual cue indicating "inactive" services or organizations returned on a search, and the ability for users to make multiple selections of organization and/or services to further investigate. Also to be consistent with the SPARCRequest module, the breadcrumb of the returned search results now displays to show where the organization/service belongs in the tree structure.

47. (SPARCCatalog) Status Options Section Display Improvement
Editable and Available statuses are now displayed side-by-side in table format for ease and clarity.


48. (SPARCCatalog) Error Message Improvement
The visibility of error messages in Catalog has been enhanced and also includes more detail describing the error that has occurred.

49. (SPARCReport) Short Interactions Canned Report
A new report was contributed by LA CaTS that includes all short interaction details entered by service providers, based on the selected criteria. The following information is included in the report:

Service Providers, [First Available] Provider/Program, Date, Subject of Interaction, Type of Interaction, Duration in minutes, Investigator Name, Investigator Email, Investigator Institution, Conversation Notes.

The Short Interactions report is configurable for OS Partners (i.e., the Short Interaction button is only visible in SPARCReport if 'use_short_interaction' is turned on; it is set to false by default).


50. (SPARCForms)
View Improvements: Submitted By, URL, Text Wrapping
The following improvements have been made to the SPARCForms view:

  1. URL has been added on the SRIDs (1st column), to link out to the corresponding Protocol page on SPARCDashboard (opens in a new tab);

  2. “By” Column to display the person who submitted the response;

  3. Content wrapping so that word breaks do not occur;

  4. 'Complete' column defaulted to be hidden (users are able to add this column via toolbox)

51. (SPARCForms) SPARCForm Login Bug
A previous routing error occured after clicking SPARCForms while logged out. Navigation links are now only displayed to the base modules; (i.e., SPARCRequest and SPARCDashboard), when logged out; SPARCFulfillment, SPARCCatalog, SPARCReport, SPARCFunding, and SPARCForms buttons on NavBar are only displayed to authorized users after logging in.

52. SPARCForms) New Form Creation Bug
There was a previous bug that prevented an authorized user to create a new form. An error message displayed, and then a new blank form that would eventually display in the forms with the ability to edit. This bug has since been fixed.

53. (SPARCCForms) Save Filter Bug

There was a previous bug that did not allow a filter to save without a start or end date. This bug has since been fixed.

54. (SPARCForms) View Response Likert Bug
When viewing a user's Survey or Form response with a Likert scale, the user was previously able to click other Likert options' labels and change the selected radio button. Data was not being affected, but the radio buttons are no longer selectable/changeable to avoid confusion.


55. (SPARCFunding) Created (Date) Column Sorting
Dates were previously sorting based on month instead of year, and were not sorting based upon chronological order. This bug has since been fixed.



56. (SPARCFunding)
Document Download Table Improvements
The following improvements have been made to download table:

  1. The SRID column currently was previously a link to the corresponding Admin Edit page. This link has been removed and now also sorts on page load.

  2. The SRID was previously sorting the actual URL rather than the display SRID.

  3. In addition to the link to Admin Edit page, a link has been included to the Protocol page on Dashboard.

  4. An "Actions" column has been added that displays two buttons: Protocol (green) and Admin Edit (orange).

  5. Ability to export the table to excel.

  6. Ability to select/deselect columns.

57. (SPARCFunding)  Refresh Glyphicon on Pre-Application/LOI and Application Buttons on Document Download page.
In this production, a refresh icon was added on both Pre-Application and Application buttons. Previously on the funding Document Download page, the selected documents table was "Pre-Application/LOI" on page load. When a funding Admin switched to the "Application" table, then clicked on the link to the Admin Edit page for a selected service request and changed the status to either Application Submitted, Rejected, or Deadline Missed,etc., the service request status on the Application Documents table was not being updated. The user had to reload the entire page, and click the "Application" button again in order to see the updated data .

Users can now click the refresh icon to show updated data instead of reloading the entire page and switching to the "Application" documents table.

SPARCRequest Rake Tasks and Setting Changes

  • ENV['SECRET_KEY_BASE'] needs to be set on production;

  • time_zone=Eastern Time (US & Canada) needs to be set in .env file for all environments;

  • Rake migrate_ids_to_bigint;

  • Rake data:clean_up_past_statuses (optional, only necessary if there is historical status label inconsistency).

SPARCFulfillment Version 2.8.0 New Features


1. (SPARCFulfillment) Security Alert - Loofah Gem

The Loofah gem has been updated to v2.2.1 in this production after a security vulnerability alert was received from GitHub.

2. (SPARCRequest) Dependabot Integration

In previous releases (since 2017), the Hakiri Facets security scan has been used to check on any necessary gemfile updates. These updates were performed to avoid potential system vulnerabilities. To improve this process, Dependabot has been integrated into the SPARCFulfillment code base, which checks the gem updates with dependency and security issues and generates a pull request automatically to the repository when an gem update is suggested.


3. (SPARCFulfillment) Homepage Pagination
Before this production, SPARCFulfillment homepage was a long list of all the requests in which a logged-in user had access. When the user had access to more than one organizations, or had a lot of requests, it was taking a long time to load the page. In this production, SPARCFulfillment homepage  has been paginated (i.e contents divided onto different pages), while applying the same general styling as SPARCDashboard for consistency. The search function and other toolbox functions on top of the table are still working as before, and the search box will still query through all requests.

4. (SPARCDashboard) Go to Fulfillment Link should Directly Link to the Corresponding Request Page

Previously, after a request had been pushed to SPARCFulfillment and the user clicked the "Go to Fulfillment" button, a new tab opened showing the SPARCFulfillment homepage. For better efficiency, this has been changed to link directly to the corresponding request/protocol in SPARCFulfillment.

5. (SPARCFulfillment) Export to MS-Excel Bug
Previously, an error occurred when attempting to "Export" the records listed in your current view to an MS- Excel file. This bug as since been fixed.

 

6. (SPARCFulfillment) Add Funding Source Column to Fulfillment/Procedures Level

Previously in SPARCRequest, users could change/update the funding source of a protocol at any time point. If that occurred after a clinical or non-clinical service had been fulfilled in SPARCFulfillment, it could have caused cost discrepancies because of the difference in funding source at a different time point. In this production, a funding source column has been added to the fulfillments/procedures level to maintain the record of funding source at the time of fulfillment, and the Funding Source column has been added to the Invoice Report as well.



7. (SPARCFulfillment) Add Invoiced Flag and Lock Down Fulfillments That Have Been Invoiced 
Some SPARCFulfillment users have been using the Invoice Report to send out their bills. In this production and for better invoicing practice, an Invoiced flag column has been added on the fulfillments/procedures level, respectively, to indicate invoiced fulfillments. When the invoiced flag is tagged to be true (or 1), the corresponding fulfillments are locked down and are not allowed to be deleted or revised in SPARCFulfillment.


8. (SPARCFulfillment) Users are Able to Change Procedures Status on Un-started Visits

A previous bug showed it was still possible to alter the state of a procedure by clicking within the disabled fields, even though procedures inside of un-started visits appeared to be disabled.  This has been fixed to show an error message indicating that visits must be started before any procedures are able to be altered. 


9. (SPARCFulfillment) Reset Visit Function Improvement

The "Reset Visit" button on the Participant Tracker section of SPARCFulfillment was only resetting the visit back to the same list of services as when the visit was built (when the user clicked on the visit from the "Select a Visit" the first time). This caused issues when the user would change the master calendar on the "Clinical Services" tab in between and make new selection(s) on that visit, resulting in the updated selection to not be reflected.


When the service was added/deleted as an active service, users could still go in and add/delete the service on individual visit, but that was very inconvenient, and not good for data consistency. In this production, the "Reset Visit" function has been reconfigured to look back to the master calendar and pull in new selected services from the calendar that don't already exist on this visit. The function will also hide services that have been deleted on the master calendar on that visit.


10. (SPARCFulfillment) Add RMID Column to Fulfillment Reports
After RMID was added to the Invoice Report of SPARCFulfillment, Clinical Providers asked for RMID to be consistent showing across all Fulfillment reports. The RMID column has been added into all Fulfillment reports: Auditing Report (clinical & non-clinical), Project Summary Report, and Visit Report. The RMID columns have been linked with the RMID configuration in SPARC.

  

11. (SPARCFulfillment) Reports Date Viewed Time Format Bug
After the last production release, a bug was discovered in the "Date Viewed" column on Fulfillment Reports page. The time format was inconsistent when compared with previous Date Viewed or Date Created columns (i.e. 3:55 PM is shown as 15:58 PM); this has been corrected and should display consistently.


12. (SPARCFulfillment) Audit Report Query Improvement

The Audit Reports in SPARCFulfillment were previously taking a long time to run, especially for larger queries (1,000+ protocols).  Improvements have been made to significantly increase the speed of these queries, so that even larger reports can be produced within a few minutes. 



13. (SPARCFulfillment) Invoice Report Conditional RMID Column
Conditions were added to the RMID column in the Auditing, Project Summary, and Visit reports, and now they are also added to the Invoice report.


14. (SPARCFulfillment) Klok Import Error Message Refactor
The error message for Klok imports was previously hard-coded, which made inclusion of different error language difficult. For example, the error message when the pricing map validation failed was showing the wrong error message about user format. In this production, the error messages were refactored with the appropriate language for the pricing map validation failure, and a report is no longer generated when an error happens (and the import didn't initiate). Previously a report file was generated, but when clicked upon, it could not be found.



SPARCFulfillment Rake Tasks and Setting Changes

  • (IN TMUX) bundle exec rake data:find_historical_funding_sources; For Fulfillment new feature #6

List of Programming Changes with Links to GitHub

SPARCRequest v3.3.0:


SPARCFulfillment v2.8.0:

Copyright © 2011-2020 MUSC Foundation for Research Development