Request for Proposal for Software Quality

Assurance and Validation Services for

the Legacy Replacement Project

Issued by:
New York City Employees’ Retirement System

March 13, 2020

 Software Quality Assurance and Validation Services for the LRP

Proposal Reference ID#: 04282020-LRP

Proposal Release Date: March 13, 2020

Dear Prospective Respondent:

The New York City Employees’ Retirement System (“NYCERS”) is initiating a Request for Proposal
(“RFP”) for comprehensive Software Quality Assurance and Validation Services for the NYCERS
Legacy Replacement Project (LRP).
The determination for award shall be made to the responsive and responsible proposer(s) whose proposal
is determined to be the most advantageous to the New York City Employees’ Retirement System taking
into consideration the price and other factors or criteria that are set forth in this RFP.

RFP ID # 04282020-LRP Page i

 Software Quality Assurance and Validation Services for the LRP

Table of Contents
A. Executive Summary ............................................................................................................................. 1
A.1. Project Expectations ............................................................................................................... 1
A.2. Responsiveness ..................................................................................................................... 2
A.3. Standardized Response Format ............................................................................................. 3
A.4. Definitions ............................................................................................................................... 3
B. Background and Overview................................................................................................................... 5
B.1. LRP Governance and Risk Management ............................................................................... 6
C. Organizational Capability and Relevant Experience ............................................................................ 9
C.1. Financials ............................................................................................................................... 9
C.2. Customer History .................................................................................................................... 9
C.3. References ............................................................................................................................. 9
D. Scope of Services .............................................................................................................................. 10
D.1. Independent Testing Strategy and Plan................................................................................ 12
Test Environments .................................................................................................. 13
D.2. Test Preparation and Execution............................................................................................ 13
Test Scripts, Cases, and Scenarios ........................................................................ 13
Testing Tools........................................................................................................... 14
Automated Testing .................................................................................................. 14
Test Evaluation ....................................................................................................... 14
Defect Management and Remediation .................................................................... 15
D.3. Reporting and Metrics........................................................................................................... 16
D.4. Testing Support .................................................................................................................... 16
D.5. Project Team Training .......................................................................................................... 17
D.6. Knowledge Transfer.............................................................................................................. 17
D.7. Project Management of the Software QA and Validation Services Engagement .................. 18
Project Management Tools ..................................................................................... 18
Project Plan and Schedule Management ................................................................ 18
Risk and Issue Management ................................................................................... 21
Project Reporting and Metrics ................................................................................. 21
E. Project Staffing and Resource Management ..................................................................................... 22
E.1. Project Organization ............................................................................................................. 22
E.2. Respondent Project Staffing ................................................................................................. 22
Key Personnel ......................................................................................................... 24
Supporting Roles..................................................................................................... 25
F. Proposal Components and Requirements ......................................................................................... 26
F.1. Technical Proposal ............................................................................................................... 26
F.2. Cost Proposal ....................................................................................................................... 27
F.3. RFP Exceptions .................................................................................................................... 27

RFP ID # 04282020-LRP Page ii

 Software Quality Assurance and Validation Services for the LRP

F.4. Additional Proposal Templates ............................................................................................. 28

F.5. Insurance Policies................................................................................................................. 28
G. Procurement Process and Requirements .......................................................................................... 30
G.1. Designated Contact .............................................................................................................. 30
G.2. Procurement Milestones and Anticipated Timings ................................................................ 30
G.3. Written Respondent Questions ............................................................................................. 31
G.4. Responses to Written Vendor Questions .............................................................................. 31
G.5. Receipt of Respondents’ Electronic and Hard Copy Proposals ............................................ 31
G.6. RFP Responsiveness and Initial Review of Written Proposals ............................................. 32
G.7. Respondent Presentations and Demonstrations................................................................... 32
G.8. Contract Negotiations ........................................................................................................... 33
G.9. Final Evaluations .................................................................................................................. 33
H. Terms and Conditions........................................................................................................................ 34
H.1. Multi-Term Award ................................................................................................................. 34
H.2. NYCERS Project Staffing ..................................................................................................... 34
H.3. Staff Replacements .............................................................................................................. 34
H.4. Warranty ............................................................................................................................... 35
H.5. Cost to Propose and/or Negotiate......................................................................................... 35
H.6. Collusion or Fraud ................................................................................................................ 36
H.7. Rights to Modify or Withdraw RFP ........................................................................................ 36
H.8. Travel and Other Expenses .................................................................................................. 36
H.9. Respondent Pricing .............................................................................................................. 36
H.10. NYCERS Prescribed Tools and Software ............................................................................. 36
H.11. Respondent Proprietary Tools and Software ........................................................................ 36
H.12. Ownership of Work Products and Intellectual Property......................................................... 37
H.13. Source Code Escrow/ Software Requirements ..................................................................... 37
H.14. Agreement ............................................................................................................................ 38
H.15. Change Requests ................................................................................................................. 38
H.16. Assumptions ......................................................................................................................... 38
H.17. Subcontractors ..................................................................................................................... 39
H.18. Procurement Flexibility ......................................................................................................... 39
H.19. Termination following the Project Assessment Phase .......................................................... 39
H.20. Termination without Cause ................................................................................................... 40
H.21. Permissible Work Locations ................................................................................................. 40
H.22. Firm Fixed Price for Complete Delivery of Scope and Requirements ................................... 41
H.23. Required Deliverables .......................................................................................................... 41
H.24. Deliverable Expectations Document ..................................................................................... 41
H.25. Final Deliverable Review ...................................................................................................... 42
H.26. Payments.............................................................................................................................. 42
Project Assessment Phase ..................................................................................... 42
Phased Solution Releases ...................................................................................... 42

RFP ID # 04282020-LRP Page iii

 Software Quality Assurance and Validation Services for the LRP

SLA Retainage ........................................................................................................ 43

H.27. Off-Shoring ........................................................................................................................... 43
H.28. Vendor’s Acknowledgment of NYS Freedom of Information Law ......................................... 43
H.29. RFP Inconsistencies ............................................................................................................. 43
H.30. City of New York Prompt Payment Policy ............................................................................. 43
H.31. Iran Divestment Act Compliance Rider for New York City Contractors ................................. 44
H.32. (2) The City makes a determination that the goods or services are necessary for the City to
perform its functions and that, absent such an exemption, the City would be unable to obtain
the goods or services for which the contract is offered. Such determination shall be made in
writing and shall be a public document.Doing Business Data Form Notice .......................... 44
H.33. “Appendix A” General Provisions Governing Contracts for Consultants, Professional,
Technical, Human and Client Services – Notice to Proposers .............................................. 45
I. Evaluation and Contract Award ......................................................................................................... 46
I.1. Evaluation Procedures.......................................................................................................... 46
I.2. Evaluation Criteria ................................................................................................................ 46
I.3. Basis of Award ...................................................................................................................... 46

List of Figures
High-Level LRP Governance Structure .................................................................................. 6

List of Tables
Table 1. Summary of Software QA and Validation Team Target Outcomes ........................................ 1
Table 2. NYCERS High-Level Statistics ............................................................................................... 5
Table 3. LRP Role Descriptions ........................................................................................................... 7
Table 4. Required Testing Types ....................................................................................................... 11
Table 5. Test Script, Case, Scenario.................................................................................................. 14
Table 6. Defect Severity Level Definitions.......................................................................................... 15
Table 7. Defect Complexity Level Definitions ..................................................................................... 15
Table 8. Defect Priority Level Definitions ........................................................................................... 16
Table 9. Blackout Periods .................................................................................................................. 20
Table 10. Respondent Proposed Staff Required Classifications .......................................................... 22
Table 11. Mandatory Key Personnel .................................................................................................... 24
Table 12. Procurement Milestones....................................................................................................... 30
Table 13. Evaluation Criteria ................................................................................................................ 46

All appendices and response templates listed below are available on the NYCERS Procurement Portal:
https://nycers.procureport.com

RFP ID # 04282020-LRP Page iv

 Software Quality Assurance and Validation Services for the LRP

Appendices
Appendix 1. Required Deliverables
Appendix 2. Deliverable Expectation Document
Appendix 3. Statement of Work
Appendix 4. Service Level Agreements
Appendix 5. Iran Divestment Act Compliance Rider For New York City Contractors
Appendix 6. Doing Business Data Form Notice
Appendix 7. “Appendix A” General Provisions Governing Contracts For Consultants, Professional,
Technical, Human And Client Services
Appendix 8. NYCERS Supplemental Provisions Governing Software Quality Assurance and Validation
Services for Technology Projects

Response Templates
Template A. Customer History
Template B. Respondent Firm Customer References
Template C. Technical Proposal
Template D. Cost Proposal
Template E. Respondent Exceptions to this RFP
Template F. Vendor Information Form
Template G. Vendor Acknowledgement of NY State Freedom of Information Law
Template H. Acknowledgement of Addenda
Template I. Bidder’s Certification of Compliance with Iran Divestment Act
Template J. Doing Business Data Form
Template K. Affirmations and Broker’s Forms
Template L. Responsiveness Certification Form
Template M. Vendor Written Questions

RFP ID # 04282020-LRP Page v

 Software Quality Assurance and Validation Services for the LRP

A. Executive Summary
The New York City Employees’ Retirement System (“NYCERS”) is embarking on a large-scale
modernization to optimize its ability to administer retirement benefits for New York City employees and
provide new and innovative services. This project is referred to as the NYCERS Legacy Replacement
Project or “LRP.” NYCERS previously issued a Request for Proposal (RFP) to solicit proposals for the
implementation of a pension administration solution that will satisfy LRP strategic objectives and goals.
NYCERS is issuing this RFP to solicit proposals for Software Quality Assurance (QA) and Validation
services to ensure the new solution’s software code is of the highest possible quality and meets business
specifications.
A summary of goals for this Software QA and Validation engagement are outlined in Table 1 below.
Table 1. Summary of Software QA and Validation Team Target Outcomes

Goal 1 Goal 2 Goal 3

Ensure High Quality Software Testing Knowledge Transfer to Ensure Accountability for System
that Meets Business NYCERS Testing
Specifications
• The Software QA and • The Software QA and • The Software QA and
Validation Team leverages Validation Team includes Validation Team is
expertise to validate System NYCERS personnel in all independently accountable for
Integrator’s (SI’s) testing aspects of test planning and certifying that system tests are
approach, methodology, and execution, including automated comprehensive, accurate, and
plans testing complete
• The Software QA and • The Software QA and • The Software QA and
Validation Team conducts all Validation Team ensures Validation Team provides
necessary testing to ensure a NYCERS personnel can supporting evidence when
stable, secure system that is effectively employ all utilized issuing their recommendation
free of defects. This testing testing tools and methodology for acceptance of test results
should go above-and-beyond approach without assistance
simply mirroring SI tests • NYCERS retains all testing
• The Software QA and artifacts (e.g., test scripts,
Validation Team performs plans, etc.) post-
additional testing after SI implementation and is able to
conducts defect remediation execute and build upon this
and system updates library
• The Software QA and
Validation Team ensures
traceability between tests and
system requirements

A.1. Project Expectations

NYCERS expects Respondents to address the following general expectations throughout their proposal:

RFP ID # 04282020-LRP Page 1

 Software Quality Assurance and Validation Services for the LRP

• Respondent’s testing approach and plan must be comprehensive and rigorous and exceed SI
testing where necessary.
• Respondent’s approach and plan for system testing must exhibit significant flexibility to ensure
consistency with the SI’s approach and plan for system testing based upon a phased
implementation of the new pension administration solution and support the decision-making
process in software acceptance.
• NYCERS will prioritize risk management throughout the project and will be supported by multiple
parties. Respondents are expected to fully embrace and account for this approach to risk
management in their proposal. The vendor is expected to demonstrate its commitment to
coordinating with other NYCERS third-party contractors to support effective risk management
activities throughout the project.
• The vendor must utilize industry-leading, commercially available, non-proprietary automated testing
tools that NYCERS may adopt at the completion of the LRP.
• The vendor must utilize industry best practices in their testing approach and methodology and is
required to support NYCERS’ transition in adopting this approach.
• The vendor must use NYCERS’ project management tools (e.g., Planview, see Section D.7.1) to
ensure there is a clear “single source of truth” regarding all aspects of the project. These tools must
provide NYCERS stakeholders with full visibility into all project activities at any time.
• The vendor should leverage relevant prior work and testing artifacts to efficiently conduct all testing
activities.

A.2. Responsiveness
NYCERS will only consider proposals where the Respondent (i.e., prime contractor) meets on their own
without the need to rely on any subcontractors or other partners all of the responsiveness requirements.
Respondents and proposals that do not meet the criteria will be deemed non-responsive. All certifications
below must be provided using the Responsiveness Certification Form (Template L):
• All Respondents must have been in business for the past 10 years.
• Respondents, as a firm, must be entirely focused on independent software testing or, alternatively,
have independent software testing as a core offering.
• Respondents must have provided software QA and validation services involving complex testing of
ERP-like systems for at least 10 distinct organizations of equal or greater size as NYCERS
(100,000 clients or more).
− Enterprise resource planning (ERP) is defined as the ability to deliver an integrated suite of
business applications. ERP applications automate and support a range of administrative and
operational business processes across multiple areas, including line of business, customer-
facing, and administrative aspects of an enterprise. ERP deployments tend to be complex.

RFP ID # 04282020-LRP Page 2

 Software Quality Assurance and Validation Services for the LRP

• Respondents must not be a contractor or subcontractor for NYCERS in another capacity relating to
any other aspect of the LRP, and must not be any vendor who is submitting any proposal related to
the LRP.
− The selected vendor will be precluded from pursuing or being engaged with NYCERS in
capacities involving the LRP beyond software QA and validation services for the entirety of
their contract with NYCERS.
NYCERS will solely determine if a proposal is responsive. Proposals deemed non-responsive will not be
considered for an award (see Section G).

A.3. Standardized Response Format

NYCERS requires all Respondents to use NYCERS’ prescribed templates and formats, including the main
technical proposal in Microsoft Word format and provided with this RFP. Additional information about
proposal components and requirements is available in Section F.

A.4. Definitions
Below are definitions of some recurring terms used throughout the RFP:
• Client: an individual external User, typically an Active NYCERS Member or a Retired Client. Clients
may also be individuals associated with NYCERS Members or Pensioners, such as a beneficiary.
• Employers: any City agencies or entities who employ individuals who are either eligible to join, or
are required to join, NYCERS’ pension plan(s).
• NYCERS Stakeholder: any individual, internal or external, who has a personal or business interest
in NYCERS as an organization and, therefore, directly or indirectly the LRP project (e.g., clients,
staff, Board of Trustees).
• User: any individual, internal or external, who will utilize the LRP solution.
• NYCERS staff: individuals working at NYCERS. NYCERS at its sole discretion reserves the right
to include contractors as staff.
• Respondent: the prime contractor responding to this RFP. The Respondent must be a single firm
proposing to provide the solution and services described herein, either directly or through
subcontractors under its direct supervision.
− The Respondent (i.e., prime contractor) may not be a partnership of firms (e.g., joint venture).
In the case of multiple firms proposing to work together, one must declare itself to be the prime
contractor (i.e., Respondent) with all other involved firms being formal sub-contractors to the
prime contractor.
− NYCERS will hold the prime contractor solely responsible and liable for all its subcontractors.
It is the prime contractor’s responsibly to ensure its agreements with subcontractors are
appropriate to this end and enable full compliance with all NYCERS RFP terms and conditions
and any subsequent agreements (e.g., Section H.17).

RFP ID # 04282020-LRP Page 3

 Software Quality Assurance and Validation Services for the LRP

− The Respondent must on its own without consideration of any subcontractors meet all
responsiveness criteria in Section A.2.

RFP ID # 04282020-LRP Page 4

 Software Quality Assurance and Validation Services for the LRP

B. Background and Overview

In addition to the content provided in this section, please refer to the Request for Proposal for IT Services
Pension Administration Software and System Integration Services for the Legacy Replacement Project,
available here: https://a856-cityrecord.nyc.gov/RequestDetail/20190401032 for additional background
information on NYCERS, including: strategic vision and objectives, business divisions, pension plan
offerings, and current technology landscape. Respondents should be sure to pay particular attention to
Appendix 3 – Salesforce CRM Implementation Reference of the Request for Proposal for IT Services
Pension Administration Software and System Integration Services for the Legacy Replacement Project.
The NYCERS’ Salesforce CRM will be integral to the LRP solution and the Software Quality Assurance
vendor’s scope of responsibility.
The entire Request for Proposal for IT Services Pension Administration Software and System Integration
Services for the Legacy Replacement Project package, linked above, is considered part of this RFP and
must be reviewed and appropriately referred to and accounted for by all Respondents in their proposal.
NYCERS was founded by the New York State Legislature in 1920 with 13,331 members. Today, NYCERS
has grown into the largest municipal public employee retirement system in the United States with over
350,000 active members and retirees, including civilian employees such as clerical workers, accountants
and social workers, and uniformed employees such as New York City Correction Officers and Sanitation
Workers.
Table 2. NYCERS High-Level Statistics

500 Employees
11 Employers
185,000 Active Members
145,000 Retired Members and Beneficiaries
26,000 Other Clients (e.g., Vested members)
Note: figures above are an approximation.
NYCERS membership is available to, and required of most, if not all, employees of the following
organizations:
• City of New York
• NYC Transit Authority
• NYC Housing Authority
• NYC Health and Hospitals Corporation
• MTA Bridges and Tunnels
• NYC School Construction Authority
• NYC Housing Development Corporation
• City University of New York – Senior Colleges

RFP ID # 04282020-LRP Page 5

 Software Quality Assurance and Validation Services for the LRP

• NYC Municipal Water Authority

For additional information, refer to page 199, Table 12 of the 2019 Comprehensive Annual Financial
Report (CAFR) available here https://www.nycers.org/sites/main/files/file-attachments/cafr2019.pdf. 1

B.1. LRP Governance and Risk Management

Managing the risk associated with the LRP’s large and complex implementation is a top priority for NYCERS.
As such, NYCERS will employ a robust governance and risk management model for the LRP that the selected
vendor will be expected to fully support. This model is centered on the governance structure and risk
management teams described in Figure 1 and Table 3 below:
High-Level LRP Governance Structure

1 While NYC Off-Track Betting Corporation and New York State Courts are employers listed in the 2019 CAFR, they are not
listed here. This is because neither of these employers has active employees with NYCERS membership as of June 30, 2017
according to the 2019 CAFR, page 199, Table 12.

RFP ID # 04282020-LRP Page 6

 Software Quality Assurance and Validation Services for the LRP

Table 3. LRP Role Descriptions

Role Description
NYCERS Board of The Board is the governing body of NYCERS. It serves as the Fiduciary to Members,
Trustees Pensioners, and Beneficiaries. Its responsibilities include oversight of the system,
establishing the administrative budget of the system, promulgating rules and
regulations necessary to carry out provisions of law and investing the assets of the
Retirement System. In addition, the Board and the Executive Director will consider
risk mitigation strategies and adoption recommendations from the Project Risk
Consultant for material risks.
Project Risk Consultant Under the direction of the Board of Trustees, will have wide latitude to monitor and
assess project activities and outputs for the purposes of evaluating risk throughout
the project.
Executive Director The NYCERS Executive Director will be the Project Authorizer, with overall
accountability for ensuring the project is appropriately resourced and helping
oversee alignment with NYCERS vision and strategic objectives.
Deputy Executive Director The NYCERS Deputy Executive Director will support the Executive Director in
overseeing the project at an executive level.
Chief Operating Officer The NYCERS Chief Operating Officer will be the primary executive-level manager
helping oversee the project with focus on ensuring the selected Respondent delivers
the project on schedule and within budget.
Independent Validation Will advise the Executive Director, Deputy Executive Director and Chief Operating
and Verification Officer, focusing on all project aspects and helping ensure the best solution is being
Consultant built in the most effective and efficient way possible for NYCERS with minimal risk.
Chief Information Officer The NYCERS Chief Information Officer will be an executive-level manager helping
oversee the project with focus on evaluating the degree the technical elements of the
implementation are being implemented by the selected Respondent as agreed and
in a manner consistent with all SOW and contractual requirements.
Enterprise Risk The Enterprise Risk Committee’s role is to oversee project and organizational risk
Committee identification, assessment, treatment (if any), monitoring, tracking and escalation of
project risks.
PMO Deputy Director The NYCERS PMO Deputy Director ensures that the selected Respondent (i.e., SI)
is meeting all of the project requirements and escalates issues to the NYCERS
Executive Director, PMO and Chief Operating Officer. The PMO Deputy Director
works closely with all the project teams to mitigate project risks and challenges
including schedule, scope and budget. In addition, the PMO Deputy Director
coordinates with the NYCERS stakeholders (both internal and external) to ensure all
communications are disseminated in a timely manner.
NYCERS Program The Program Management Office (PMO) ensures all NYCERS project management
Management Office requirements and policies are met and will assist in coordinating the NYCERS
(PMO) Project Team, as well as with any change control triggered by requests for changes
(RFCs). The PMO will also work closely with all project risk management teams.
LRP Vendor(s) Responsible for the delivery of the project. Will primarily include the selected
Respondent and their subcontractors, however may also include other vendors and
consultants brought onto the project at the discretion of NYCERS.

RFP ID # 04282020-LRP Page 7

 Software Quality Assurance and Validation Services for the LRP

Role Description
NYCERS Project Team The NYCERS Project Team are the individuals who will be assigned by NYCERS to
work with the SI as well as any other vendors working on the project.

Any exceptions to this model, and the Respondent’s ability to participate in it fully, must be described in the
RFP exceptions (see Section F.3).

RFP ID # 04282020-LRP Page 8

 Software Quality Assurance and Validation Services for the LRP

C. Organizational Capability and Relevant Experience

Each Respondent must provide an overview of its organization and demonstrate in the submitted proposal,
the degree to which the organization is capable of successfully completing software quality assurance and
validation on a project of the scale and complexity of the LRP.

C.1. Financials
Respondents must explain the current and anticipated financial state of their organization in the submitted
proposal. Each Respondent and all subcontractors (including all subcontractors of subcontractors at any
level) must also submit a copy of their certified financial statements or audit reports with their proposal for
each of the last five (5) years (2015–2019) in addition to any material information the expected to impact
financials in the current year (2020).

C.2. Customer History

In addition to providing some customer history background within their proposal, Respondents must provide
a complete history of all final contracts involving public pension systems, both in the United States as well
as other countries, to include both projects with an original start date on or after January 1, 2007 as well as
projects ending on or after January 1, 2007 regardless of start date using Template A. All types of contracts
must be disclosed (e.g., both goods and services) with a high-level description of the scope of services
(e.g., Independent Testing). The contracts of subcontractors, including contracts where the subcontractor
was not partnered or otherwise involved with the Respondent, must also be disclosed. If the Respondent or
any subcontractors were part of any merger, pre-merger customer history with public pension systems for
all involved companies must be included in the customer history.
Template A requires specific information about contracts, such as the original and current contract value,
the nature of the contract, and the number of contract modifications. This type of information is generally
publicly available upon request, is not confidential, and may be pro-actively published (e.g.,
www.checkbooknyc.com), in many jurisdictions. Respondents may neither assume that such information is
confidential, nor ask NYCERS to determine confidentiality. If in specific cases Respondents believe the
requested information is confidential, they must perform the due diligence to confirm confidentiality and
provide the basis and rationale for confidentiality in the proposal. If NYCERS subsequently independently
investigates a Respondent’s confidentiality claim and determines the information is not confidential, the
Respondent’s proposal may be deemed non-responsive.

C.3. References
Respondents shall provide six (6) references using Template B. These references should be for
independent software quality and validation services conducted for large ERP-like system modernization
projects of similar scale and complexity as NYCERS (100,000 clients or more). These references must be
for customers that appear in the Respondent’s customer history (see Section C.2). NYCERS may also
choose to arrange site visits directly with any or all Respondent references.
Note that for references, some specific information about contracts is requested for which Respondents
must follow the requirements described in Section C.2.

RFP ID # 04282020-LRP Page 9

 Software Quality Assurance and Validation Services for the LRP

D. Scope of Services
The vendor will be responsible for delivering all services necessary to ensure the LRP solution is of the
highest quality, meets all requirements and has been designed as expected, and meets NYCERS’
expectations, in addition to other services related to testing that will support NYCERS. In their proposals,
Respondents must address:
• How the proposed testing strategy is suitable and sufficient for all required testing types (see
Table 4), and clearly highlighting nuances within any given testing type.
• Testing approach and methodology details, including:
− Comprehensive details on the proposed testing approach and methodology, including fully
articulating how the test strategy and plan that outlines the test cycles and testing details (e.g.,
test coverage, techniques, traceability) will be delivered.
− The degree the proposed approach and methodology has been tailored for NYCERS and the
degree it is based in proven, industry-leading practices and methodologies appropriate for
projects of the scale and nature of the LRP.
− How the Respondent’s team will function independently, as well as act as an extension of the
NYCERS staff, and will be directly responsible for performing independent software testing as
validation of the testing performed by the SI.
− Details on all material qualitative and quantitative elements of the proposed testing approach
and methodology.
− Clarity on entry and exit criteria for all testing types
− Discussion of the actual execution of testing and how it will be managed and reported on
against entry and exit criteria for the testing phases
• How it will ensure that independent testing activities stay aligned with SI’s project plan:
− The SI will take a phased implementation approach for the new solution. The SI is under
procurement and therefore the exact phased implementation approach is not yet confirmed
(refer to the Request for Proposal for IT Services Pension Administration Software and System
Integration Services for the Legacy Replacement Project for more details).
− Respondents must propose a flexible approach that can adapt as needed.
• Alignment with an Agile or Agile-like LRP SI solution implementation approach.
− NYCERS expects an Agile or Agile-like implementation to be employed by the LRP SI vendor
no matter which vendor(s) are awarded.

RFP ID # 04282020-LRP Page 10

 Software Quality Assurance and Validation Services for the LRP

Table 4. Required Testing Types

Testing Type Description

Integration Testing • Testing of higher-level system components as modules and other solution
units are integrated by the development team (e.g., Customer Relationship
Management solution).
System Testing • Comprehensive testing of the complete integrated system release.
Parallel Testing • Uses a common set of inputs to assess the compatibility of outputs from the
new system with the outputs from the legacy system.
External Interface Testing • Ensures proper function of interfaces with systems owned and managed by
external organization outside of NYCERS that relate to the solution (e.g.,
employer data exchanges).
• Includes closely coordinating and collaborating with staff of external
organizations to the extent directed by NYCERS.
Performance Testing • Determines how the system (including computers, networks, software
programs or devices) performs in terms of responsiveness and stability under
different workloads primarily to help measure, validate or verify quality
attributes such as scalability, reliability, interoperability and resource usage,
as well as locate bottlenecks.
• Includes load testing and stress testing.
• Verifies testing-related SI SLAs will be met.
Backup, Failover, Roll-back • Ensures the solution meets all requirements for backup, failover, recovery,
and Recovery Testing disaster recovery, and business continuity.
• Includes testing the ability to roll-back the solution from its current release to
a previous release, and to legacy systems.
• Note: NYCERS may conduct backup, failover, roll-back and recovery testing
independent of the project as a course of normal business. The vendor will
be required to participate in such testing, including helping NYCERS to
develop a strategy and plan for execution of the testing and providing
feedback on test progress. The vendor may also need to provide staff
augmentation during test execution.
Data Migration Testing • Ensures all functional and non-functional aspects of the application are met
after migration to the new system, with data integrity and no loss of data
• Includes mock conversions and migrations
Security/Penetration Testing • Provides visibility into security vulnerabilities that could be leveraged by
malicious users or others and require remedy.
• Note: Respondents should highlight the experience, skills, and certifications
they possess in regards to security/penetration testing.
Regression Testing • Ensures faults have not been introduced/uncovered as a result of the
changes, or introduced errors or regressions in the unchanged code.

RFP ID # 04282020-LRP Page 11

 Software Quality Assurance and Validation Services for the LRP

Testing Type Description

User Acceptance Testing • Validation of end-to-end business processes and intended solution function
(UAT) from an end-user perspective.
• Note: it is planned for NYCERS staff to perform UAT under the direction of
the SI. NYCERS intends to carry out a substantial UAT effort, performed by
NYCERS staff at all levels in the organization for all functions and processes
of the solution, end-to-end. The vendor will be required to participate in such
testing, including helping NYCERS to develop a strategy and plan for
execution of the testing, creating additional test suites as necessary, and
providing feedback on test progress.
Post-Deployment (“Smoke”) • Ensuring the release has been successfully deployed to production and that
Testing it is behaving as designed or otherwise intended once deployed.

D.1. Independent Testing Strategy and Plan

The vendor will be responsible for developing a comprehensive and detailed strategy and plan for
conducting independent testing. This work will happen concurrently with LRP Project Assessment Phase
(“Phase 0”) as described in the Request for Proposal for IT Services Pension Administration Software and
System Integration Services for the Legacy Replacement Project (see Section B). Development of this
plan must be completed during the Project Assessment Phase, which is ten (10) weeks. In their proposals,
Respondents must address:
• The underlying framework that will guide development of the strategy and plan.
− Be sure to describe any industry standards (or similar) that will be used.
• The value the Respondent will provide with their services and how this will be demonstrated.
− For example, how the Respondent’s plan will provide true independent validation of system
integration testing, help ensure production readiness and reduce overall project risk.
• Facilitating a series of workshops with NYCERS during project initiation to cover topics including,
but not limited to:
− Determining and validating key requirements for the strategy and plan, including critical
success criteria.
− Discussing and refining draft versions of the strategy and plan.
− Validating the strategy and plan.
− Discussing and validating testing environment requirements.
− Discussing and validating the plan, approach, and artifacts required to support NYCERS
decision to formally accept software code.
• Aligning the progression of testing activities with the SI’s phased implementation approach.
• Approach and methodology for risk and issue identification and mitigation.

RFP ID # 04282020-LRP Page 12

 Software Quality Assurance and Validation Services for the LRP

Test Environments
The Respondent will propose an approach for establishing and utilizing test environments (including
hardware and software) for all testing types and for all project phases. Respondents must discuss in their
response:
• How they have handled the management of test environments on previous independent testing
projects, highlighting the most successful strategies.
• If the approach would be to establish and maintain separate test environments from the SI what
controls would be put in place to ensure uniformity between the test environments, or if the
approach would be to utilize the test environments provided and configured by the SI, how the test
environments would be efficiently shared between both parties to prevent any project delays.
• Any test environments that could not be hosted by a commercially available cloud provider of
NYCERS choosing and why not.

D.2. Test Preparation and Execution

NYCERS expects the vendor to conduct all preparation and execution of testing activities necessary to
validate testing completed by the SI, as well as additional testing deemed necessary to ensure the
implemented solution is stable, secure, and meets all of NYCERS business requirements.

Test Scripts, Cases, and Scenarios

The execution of a comprehensive set of test scripts, cases and scenarios across all required testing types
(see Table 4) is essential. The Respondent will be responsible for validating all test scripts, cases and
scenarios developed and executed by the SI, as well as developing and executing additional tests
necessary to ensure a stable, secure system that meets all business specifications. Respondents must
provide examples of prior test scripts, cases and scenarios as well as describe the development
methodology for each. Respondents must discuss their approach to ensuring accurate results are
achieved from the tests. Tests are expected to be designed against an appropriate industry standard and
should have characteristics such as:
• Clear association with the underlying the independent testing strategy and plan.
• Clear traceability to requirements, design, and other appropriate system elements.
• Alignment between test scripts, cases, and scenarios (see Table 4).
• Clear understanding of dependencies between tests.
• Clear identification of all inputs required to execute tests and all outputs required to verify tests
(e.g., data, conditions, valid and invalid responses, etc.).
• Clear descriptions of any required technical environment components and/or configurations
necessary to execute the tests.

RFP ID # 04282020-LRP Page 13

 Software Quality Assurance and Validation Services for the LRP

Table 5. Test Script, Case, Scenario

Testing Term Description

Test Script • Line-by-line description of actions and data needed to perform a test
Test Case • Description of a specific idea that is to be tested without detailing the exact
steps to be taken or data to be used
Test Scenario • Description of an objective a user might face when using the application

Testing Tools
The Respondent must utilize enterprise-grade, industry-leading, commercially available, non-proprietary
testing tools appropriate for a project of this scale and complexity, for all tests conducted during
implementation of the new solution. Tools must be compatible with use of JIRA to enable streamlined
defect reporting. The Respondent must grant full access to and use of tools to NYCERS, including training
members of NYCERS staff to properly and effectively employ and maintain the tools, with the intent that
NYCERS will adopt the testing tools after completion of the LRP. The Respondent must submit a
comprehensive list of all testing tools utilized on previous projects, including the tool’s purpose and the
Respondent’s level of expertise (e.g., certifications) at employing each tool. Respondents are expected to
support the operation and maintenance of all utilized testing tools throughout project implementation (e.g.,
performing tool upgrades).

Automated Testing
NYCERS wants to take full advantage of automated testing in a similar manner to other leading commercial
and public organizations when implementing a solution of this scale. Regarding automated testing, all
proposals must:
• Include automation of regression testing wherever such testing will be used.
• Explain if and how the automated testing will be leveraged for other types of testing beyond
regression testing, and explain in which areas automation would provide the greatest benefit.
• Describe how the Respondent has utilized automated testing on prior software quality assurance
and validation projects as well as the associated benefits realized over alternative methods.
• Support the proposed use of automated testing with information such as the processes and
standards around automated test design and maintenance and how the proposed automated
testing approach will be effective, optimal, and is aligned with the skills and level of the
Respondent’s proposed resources.
• Provide details on the specific automated testing tools required to support the proposed automated
approach and how they will be practically utilized during implementation.

Test Evaluation
Respondents must fully articulate how they will evaluate all aspects of testing within each phase of system
implementation in their proposal. All proposals must:

RFP ID # 04282020-LRP Page 14

 Software Quality Assurance and Validation Services for the LRP

• Address specific steps necessary to evaluate each testing type listed in Table 3 (e.g., evaluating
testing results).
• Describe the tools and processes that will be used to evaluate sufficient test coverage as well as
confirm requirements traceability.
• Provide a comprehensive list of all evaluation criteria that will be utilized throughout the testing
process, including entry/exit criteria and suspension/resumption criteria for each phase and type of
testing.
• Outline the steps and events that would trigger situations where testing would be
stalled/suspended and then subsequently resumed, including how the Respondent would help
minimize potential negative impacts to the project during any such times.

Defect Management and Remediation

The vendor will be responsible for validating the SI’s defect management and remediation actions, and
must explain how this validation will occur, including execution of additional testing. The vendor will be
required to use NYCERS’ instance of JIRA (https://www.atlassian.com/software/jira) to manage and
facilitate defect reporting. JIRA must be updated as part of the defect management process and not after
the fact, which must be accounted for in the Respondent’s proposal. All defects will be reported directly to
NYCERS staff, not to the SI. In addition, the vendor will be required to use NYCERS’ prescribed defect
severity level, complexity level and priority definitions as shown in Tables 5, 6 and 7 below.
Table 6. Defect Severity Level Definitions

Severity Level Description

Severity 1: Causes complete application/solution failure or unavailability in a production environment
Critical impacting multiple users with no known workarounds acceptable to NYCERS.
Severity 2: Causes repeated, consistent failure of functionality impacting multiple users, or information
High errors/corruption, with no known workarounds acceptable to NYCERS.
Severity 3: Failure of functionality impacting one or more users but where there is a workaround
Medium acceptable to NYCERS.
Severity 4: Minor failure that does not substantially affect the use of the application/solution where
Low minimal, if any, acceptable workarounds are necessary prior to remediation (e.g., cosmetic
issue).

Table 7. Defect Complexity Level Definitions

Complexity Level Description

High Complex remediation required that may involve multiple solution components and/or require
senior level technical resources; and/or remediation requires 40 person-hours or more.
Medium Simple remediation required that typically involves one or a very limited set of non-complex
solution components and/or junior level technical resources; and remediation requires less
than 40 person-hours.

RFP ID # 04282020-LRP Page 15

 Software Quality Assurance and Validation Services for the LRP

Complexity Level Description

Low Simple remediation required that does not require specialized technical (e.g., programming)
skills and could be remediated by trained power users or system administrators; and
remediation requires less than 10 person-hours.

Table 8. Defect Priority Level Definitions

Priority Level Description

High All critical severity defects and most high severity defects.
Medium High severity defects not assigned a high priority and some medium severity defects.
Low Medium severity defects not assigned a medium priority and all low severity defects.

D.3. Reporting and Metrics

Respondents must provide testing reporting and metrics and describe their detailed approach for doing so
in their proposal, which must:
• Deliver current, on-demand testing reports, including reports that are designed to help validate the
SI’s test results.
− Validation reports which must be submitted to NYCERS at the completion of each testing
phase, detailing the tests conducted, the test results, and recommendations for next steps.
• Include any specific tools proposed to support testing reporting and metrics in addition to Planview
(see Section D.7.1) and JIRA (see Section D.2.5).
• Use of predictive test metrics and reporting where possible.
• Explain why the proposed metrics and reporting will be effective for the LRP and what best
practices the approach is based on.
• Explain how the Respondent will ensure a “single source of truth” of all their testing data (e.g., in
cases where multiple tools are being used).

D.4. Testing Support

Throughout the project, NYCERS may require the Respondent to perform additional services outside the
scope of conducting independent testing. These may include, but are not limited to:
• Evaluating the SI’s testing strategy and plan and developing recommendations for improvement
(e.g., comprehensiveness, adherence to industry best-practices, evaluation criteria)
• Evaluating the SI’s approach to testing risks and issues and developing recommendations for
improvement
• Monitoring the effectiveness of the SI’s testing processes as well as the SI’s testing progress and
providing feedback and recommendations for improvement

RFP ID # 04282020-LRP Page 16

 Software Quality Assurance and Validation Services for the LRP

• Helping NYCERS develop a strategy and plan for NYCERS-led User Acceptance Testing and
Backup, Failover, Roll-back, and Recovery Testing. The Respondent may be required to create
additional test suites to facilitate various testing types, as well as provide feedback on test progress
and recommendations to improve the quality and efficiency of testing efforts.
• Incorporating and facilitating meaningful participation of SI resources in specific, testing related
project discussions

D.5. Project Team Training

To help ensure the NYCERS staff can contribute effectively to testing-related project activities, the selected
Respondent will be required to train NYCERS staff in areas including, but not limited to:
• The approach and methodology that will be employed by the selected Respondent to conduct
software quality assurance, as well as the use of testing tools.
• Assisting NYCERS staff in understanding the system integrators testing approach and
methodology.
NYCERS staff to be trained will be a mix of staff of various skill level including:
• Business staff, approximately 50 different individuals who will support testing activities, such as
UAT, at various times throughout project implementation.
• Technical staff, approximately 50 different individuals who will support testing activities, such as
review and validation of testing results, at various times throughout project implementation.
All proposed training approaches must include:
• Defining clear and measurable training objectives and providing the means and processes to
objectively determine if those objectives have been met.
• Remediation and make-up training.
• Direct training of NYCERS staff as a primary training method (i.e., direct interactions between
NYCERS staff and the staff of the selected Respondent, and not “train the trainer”).
− Use of training methods to complement direct training are encouraged (e.g., computer based
training).
• Providing new and refresher training throughout the LRP implementation and at all critical junctures
(i.e., not disproportionally providing training just once at the beginning of the project or at the
beginning of the first testing phase).

D.6. Knowledge Transfer

Knowledge transfer is key in NYCERS’ ability to independently conduct all types of system testing post-
implementation. NYCERS expects all proposed knowledge transfer approaches to:
• Provide continuous knowledge transfer throughout the project in a way that does not disrupt project
activities or otherwise risks, or causes, any project delays.

RFP ID # 04282020-LRP Page 17

 Software Quality Assurance and Validation Services for the LRP

• Enable direct collaboration between the actual members of the Respondent’s team performing the
work and NYCERS staff, especially those identified as being responsible for testing activities post-
implementation once vendors depart (i.e., not train-the-trainer style or similar indirect models).
• Be tailored specifically to NYCERS and not generic, including creating NYCERS-specific
knowledge transfer documentation, and include a process to evaluate the success of knowledge
transfer using objective measures.
• Include a detailed plan that supports the transfer of knowledge and approach for determining a
successful transition.
Knowledge transfer will only be deemed successful if NYCERS staff is evaluated as proficient with all
aspects of planning and conducting all testing types listed in Table 3; including, but not limited to: forming a
testing approach and methodology, creation of test suites (including for automated testing), test execution
(including automated testing), expert use of testing tools, test evaluation and reporting, and test risk
identification and mitigation strategies.

D.7. Project Management of the Software QA and Validation Services Engagement

Respondents shall be responsible for providing comprehensive project management services designed to
manage and ensure the successful delivery of software quality assurance and validation services.
Respondents must propose a project management approach based in widely recognized best practices
that is tailored to NYCERS, value-added, pro-active, collaborative, and transparent.

Project Management Tools

NYCERS will require the vendor to use its instance of Planview PPM cloud-based project management
software (https://www.planview.com/solutions-project-portfolio-management). Project governance, change
control, and reporting will take place through Planview PPM. The required defect management tool, JIRA,
is also integrated with Planview. The vendor is expected to make optimal use of capabilities enabled
through that integration.
The vendor will be expected to take full advantage of all capabilities of the NYCERS’ instance of Planview,
and must use Planview in a fashion that provides NYCERS with full real-time visibility into project status
and all activities at all times. Respondents should clearly describe in their proposal how they will use
Planview to meet these expectations.

Project Plan and Schedule Management

Respondents are to provide a detailed project plan with their proposal that demonstrates the necessary
buildup of resources and tasks to comprehensively fulfill all services listed within the RFP. NYCERS
recognizes that as the SI procurement process is not yet complete, information such as the exact number
of phased releases, as well as the scope of each release, has yet to be determined. Nevertheless,
NYCERS expects Respondents to be able to include an informative project plan based on the extensive
information provided in the Request for Proposal for IT Services Pension Administration Software and
System Integration Services for the Legacy Replacement Project (see Section B) and based on the
presumed extensive experience of the Respondent. Once a final project is agreed to with the selected

RFP ID # 04282020-LRP Page 18

 Software Quality Assurance and Validation Services for the LRP

vendor, that vendor will be responsible for adhering to the agreed upon plan throughout the project. The
project plan and explanatory information provided in the proposal should:
• Include clear work breakdown structure (WBS) information.
• Highlight key dependencies both between tasks performed by, and fully in control of, the
Respondent as well as key dependencies the Respondent’s work would have with the LRP System
Integration vendor.
• Provide information about how the Respondent would resource specific tasks, including estimated
resource loading details (quantitative).
− Resources referenced should align with the resource types and roles the Respondent provides
in their Cost Proposal.
− Respondents must be clear about what tasks are proposed to be performed on-site, off-site, or
off-shore (or some combination).
• Highlight project plan connections to key milestones and required deliverables (Appendix 1).
• Ensure the proposed project plan is conservative. Proposed project plans are to account for the
both the information available to the Respondent as well as known uncertainties. Proposed project
plans are not to reflect “best case” or “minimum” scenarios.
− NYCERS will work with the selected vendor and the LRP SI during the Project Assessment
Phase (“Phase 0”), as described in the Request for Proposal for IT Services Pension
Administration Software and System Integration Services for the Legacy Replacement Project
– refer to Section B, to finalize the project plan.
− While finalization of the project plan may result in redistribution of tasks, efforts, durations, etc.,
it may not result in an increase to the Software Quality Assurance vendor’s firm fixed price (see
Section H.22). The Respondent’s project plan submitted with this proposal must be designed
with this principle in mind.
• For all NYCERS led work proposed by the Respondent, Respondent must clearly identify all such
tasks on their proposed project plan and ensure realistic and adequate time is afforded to such
tasks, presumably informed by the Respondent’s expertise and experience. All NYCERS led work
must be thoughtfully planned and designed by the Respondent, and well-supported by Respondent
resources.
− All such tasks must each include specific Respondent staff/resource assignments with effort
hours attached to those Respondent staff/resources to demonstrate the degree the
Respondent has planned to support NYCERS led work.
− NYCERS reserves the right to require evidence from Respondents that supports the need for
NYCERS led work in a particular area, as well as evidence that substantiates a Respondent’s
proposed approach and estimates for NYCERS led work.

RFP ID # 04282020-LRP Page 19

 Software Quality Assurance and Validation Services for the LRP

− NYCERS will not lead or perform work on unrealistic schedules, or under other unrealistic
conditions, especially in any situations where the selected vendor is providing minimal support
to achieve such work in the target timeframe.
• Be provided in a format where details can be easily extracted and reviewed in detail by NYCERS
(e.g., MS Project or Excel). Project plans should not be provided in formats such as PDF.
In addition, the proposed project plan must account for the following blackout periods described in Table 9.
During blackout periods, major deployments cannot occur. Respondents should also expect reduced
availability of NYCERS staff during these times.
Table 9. Blackout Periods

Blackout Period Notes

Part of January • 5th Quarter Closing
• 1099 Run (pensioners and members with taxable loans)
Part of February • Mass Service Generation for Active Members
Part of March–April • Annual Disclosure to active members
• Annual Disclosure to vested members
All of June through the end of the • NYCERS has significant operational commitments associated with
first week of July the end of its fiscal year (June 30).
Part of September • COLA Mass Mailing
Part of October • VSF Payments
Last week of each Quarter • Certain files on the mainframe are closed to conduct quarterly runs
across all membership accounts.

The proposed project plan must be paired with an effective ongoing schedule management approach in
order to help ensure timely delivery of the project according to the plan. Respondents must demonstrate
how their schedule management approach will be pro-active and use proven methods to detect potential
schedule delays and schedule risks, early, when they are more likely to be remediated.

Furthermore, NYCERS is interested in understanding, for both potential and actual schedule delay
situations, what types of actions Respondents will commit to taking in response to actual schedule delays
to get the project back on schedule (e.g., underestimated level of effort for a given task). Along these lines,
it is important for Respondents to explain how, in these situations, they would communicate and instill
confidence in NYCERS that any schedule delays, or potential schedule details, will be resolved through the
Respondent’s actions. For a project of this scale, nature, and duration, it is essential that the vendor build
trust and a solid track record with NYCERS of predictable resolution, and minimal uncertainty around any
potential or actual schedule delays.
Respondents should anticipate and plan to resource and initiate the project within 10 days of NYCERS
issuing the Purchase Order.

RFP ID # 04282020-LRP Page 20

 Software Quality Assurance and Validation Services for the LRP

Risk and Issue Management

Respondents are expected to pro-actively monitor for, report on, and mitigate any risks and issues within
the delivery of their own software quality assurance services. Respondents are to explain in their proposal
how they would plan to do this. The selected vendor will be expected to act on risks and issues in a
reasonable and timely manner throughout the project.

Project Reporting and Metrics

The primary purpose of project reporting and metrics is to compare actual performance to the agreed upon
project plan. For example the agreed upon deliverables are furnished on time at the agreed upon level of
detail and quality. Project reporting will be required to be continuous in the sense that up-to-date
information should be in the Planview project management tool (see Section D.7.1) at all times, available to
NYCERS stakeholders to call up in Planview on demand. Respondents must ensure their proposed
approach meets this requirement.

RFP ID # 04282020-LRP Page 21

 Software Quality Assurance and Validation Services for the LRP

E. Project Staffing and Resource Management

Respondents must include in their proposal a comprehensive project staffing and resource management
plan that meets all requirements set forth in this RFP. For example, NYCERS requires the Respondent
propose specific named individuals for certain Key Personnel (see Section E.2.1). Evaluation of project
staffing and resource management will include, but is not limited to, consideration of the overall proposed
staffing and resource management plan, the named individuals, as well as the commitments the
Respondent is willing to make regarding the minimum standards of unnamed individuals that would be
assigned to the project.

E.1. Project Organization

Respondents must include in their proposal a detailed project organization chart, that clearly depicts roles,
assignments and subcontractors, aligns with the high-level project governance structure provided in B.1, and
that describes what they believe would be most effective to successfully implement the project within the
goals and expectations set forth in this RFP, including roles required of NYCERS staff, if any. Respondents
must also explain the reasoning behind their proposed project organization in their proposal. This should
include discussing the degree to which the Respondent has used similar project organization structures to
successfully deliver other comparable projects.

E.2. Respondent Project Staffing

Respondents must provide information in their proposal about every role they, and their subcontractors, will
staff on the project. This includes providing information such as the role’s responsibilities, how the role is
staffed, and the role’s minimum level of experience and qualifications, and assignment details. Key
Personnel (see Section E.2.1) have additional requirements, such as the need to name specific individuals
for all Key Personnel. In particular, NYCERS is interested in how Respondents demonstrate their level of
commitment to provide quality staffing for the project, including, but not limited to, the degree to which
proven individuals with demonstrated success conducting software quality assurance and validation on
large scale, ERP-like system modernization projects, especially in environments similar to NYCERS (e.g.,
highly regulated).

Classifying Respondent Resources

The staffing plan must classify each proposed key and supporting resource, both for key (named) and
supporting (unnamed) roles, as described in Table 10. In order to ensure consistency across proposals, all
Respondents must use these descriptions exactly as provided and may not modify or make qualifications
about them in their proposal.
Table 10. Respondent Proposed Staff Required Classifications

Resource
Description
Classification
On-Site, Off-Site, Resources must be classified as either “on-site,” “off-site,” or “off-shore.”
or Off-Shore • On-site resources are resources that will spend 75% or more of their project working
time at a NYCERS location. On-site resources must meet the following minimum
requirements:

RFP ID # 04282020-LRP Page 22

 Software Quality Assurance and Validation Services for the LRP

Resource
Description
Classification
− On-site work schedules for all resources must always be designed in the best
interests of the project.
− On-site work schedules should align with NYCERS normal business hours
whenever possible and where consistent with the best interests of the project.
− NYCERS may require at any time on-site resources to adjust their on-site working
schedule, such as to participate in project workshops or other discussions, which
will be accommodated and facilitated by the Respondent.
− On-site resources must be eligible to work in the United States for the duration of
the project.
• Resources that will spend 75% or more of their project working time at locations other
than NYCERS are to be classified as “off-site” if those non-NYCERS work locations are
within United States or Canada, and “off-shore” if those non-NYCERS work locations
are not within the Unites States or Canada. Both off-site and off-shore resources must
meet the following minimum requirements:
− Work performed off-site and/or off-shore must in no way hinder the project or delay
its progress.
− Time-zone differences may not restrict the ability for the NYCERS team to
communicate and collaborate with off-site and/or off-shore resources during
NYCERS normal business hours as needed.
− NYCERS may require at any time off-site and/or off-shore resources to appear on-
site, such as to participate in project workshops or other discussions. The timing
and duration of such on-site appearances requested by NYCERS will be
accommodated and facilitated by the Respondent at no additional cost.
− All off-site and off-shore locations where any NYCERS-related work is performed
must adhere to the minimum set of workplace standards described in Section
H.21.
− In addition, no more than one-third (33.333%) of total project resources as
identified by the Respondent in their cost proposal (see Section F.2) may be filled
by off-shore resources.
− While Respondents are permitted to propose off-shore resourcing in a manner that
conforms to the RFP, ultimately the selected Respondent’s use of any off-shore
resources is solely at NYCERS discretion and only is permitted with express pre-
approval from NYCERS.
Full-Time or Resources must be classified as either “full-time” or “part-time”:
Part-Time • Full-time resources are resources that will be exclusively assigned to NYCERS and will
work a minimum of 40 hours each week on the project, excluding approved holidays
and personal time off, for the entire assignment period.
− “Exclusively assigned to NYCERS” means a resource that will only work on the
NYCERS project and will not be assigned to, or otherwise performing work for, any
other vendor client; as well as the resource not being assigned to, or otherwise
performing, significant internal work for the Respondent or significant work for
outside entities.

RFP ID # 04282020-LRP Page 23

 Software Quality Assurance and Validation Services for the LRP

Resource
Description
Classification
• Part-time resources are resources that will be not exclusively assigned to NYCERS (as
described above) and/or will work less than 40 hours each week on the project.
• Resources that will work a minimum of 40 hours a week but that will not be exclusively
assigned to NYCERS must be classified as part-time.

Key Personnel
Specific individuals must be proposed for every Key Personnel listed in Table 11 below, and a résumé with
three (3) references be provided for each of those individuals. In addition, in order to propose a specific
individual, the Respondent must agree that the individual may not be removed from the project prior to the
start date. In the event any proposed key personnel becomes unavailable, the Respondent must take the
necessary action(s) to remedy in a timeframe that is acceptable to NYCERS. Furthermore, NYCERS shall
be entitled to damages if the loss of key staff during the project contributes to project delay (see Section
H.3).
Table 11. Mandatory Key Personnel

Role Description
Test Lead The Testing Lead will be responsible for oversight and coordination of all aspects of
system testing, including coordination and oversight of all testing cycles and testing
personnel. This individual will ensure that testing activities remain in sync with the
SI’s testing activities, and will update the Contractor’s project manager regularly
regarding the progress of testing activities. The Test Lead will provide guidance on
where and how to allocate testing resources to ensure proper execution of the
contractor’s testing approach, methodology, and plan. This individual will integrate
NYCERS QA Team members in with the Contractor’s team in order to facilitate
continuous knowledge transfer throughout the LRP. The Test Lead will confirm that
all entrance and exit criteria are met within each test phase, and will ensure that all
identified defects are properly communicated to the NYCERS staff. Specific
experience and expertise managing complex independent testing of ERP-like
systems is strongly preferred.
Project Manager The Project Manager will be the Contractor’s primary resource responsible for the
management and delivery of the project, including coordinating all Contractor and
Subcontractor project resources and ensuring such resources perform efficiently,
optimally and produce acceptable work product at all times. The Project Manager is
also responsible for ensuring continuous updating, maintenance and organization of
all project documentation. The Contractor will be required to authorize and empower
the Project Manager to make commitments to NYCERS relating to the project and
Contractor’s scope of activities.
The proposed project manager must hold one or more widely-recognized, currently
active, industry certifications in project management (e.g., Project Management
Professional).
Lead Test Automation The Lead Test Automation Engineer is responsible for overseeing deployment of test
Engineer automation solutions to increase the robustness, effectiveness, and efficiency of
testing efforts. This individual will lead the design and execution of automated,

RFP ID # 04282020-LRP Page 24

 Software Quality Assurance and Validation Services for the LRP

Role Description
reusable test scripts. The Lead Test Automation Engineer will leverage the
Contractor’s test automation framework in addition to best practices to determine
what tests (in addition to regression testing) should be automated and in what
sequence. This individual will ensure knowledge transfer of automated test
execution to NYCERS staff members.

Supporting Roles
Any role the Respondent proposes that is not a Key Personnel is a supporting role (e.g., Test Environment
Specialist). Key information on supporting roles, such as a description of the supporting role, its
responsibilities, how it will be staffed, and the minimum qualifications for the role, are required to be
provided for every supporting role proposed. Respondents are not required to name specific individuals to
supporting roles. However, if a Respondent chooses to name specific individuals to any given supporting
roles, those individuals will be subject to the same requirements and terms that apply to named individuals
in Key Personnel.

RFP ID # 04282020-LRP Page 25

 Software Quality Assurance and Validation Services for the LRP

F. Proposal Components and Requirements

This section describes the required components of a Respondent’s proposal. If NYCERS does not receive
all required proposal components, or the components received do not meet RFP requirements, the
Respondent’s proposal may be deemed non-responsive. Some portions of the proposal will require the
Respondent to use NYCERS provided templates. Each template may contain additional instructions and
requirements not listed below, that must be followed. There are further submissions that NYCERS must
receive prior to receiving the Respondent’s proposal, such as the notice of intent to bid, which are
described in Section G.
Whenever NYCERS templates are required, Respondents are responsible for fully adhering to the use and
format of the templates. Respondents are responsible for independently confirming that they can open and
use templates. Any questions regarding templates must be directed to NYCERS through the designated
contact as part of the Question and Answer period as provided in Section G.1. Respondents are only to
populate templates with the requested information and may not modify the templates to any extent for any
reason. Completed templates must be submitted back to NYCERS in their original file format and not
converted to any other file format, such as PDF. Similarly, attachments submitted by Respondents as part
of their proposal (e.g., sample deliverables) should be in their original file format wherever possible.
Proposals submitted using modified templates or the Respondent’s own templates will be deemed non-
responsive.
All appendices and response templates are available at the NYCERS procurement portal at
https://nycers.procureport.com/. Obtaining access to RFP materials requires registering an account on the
portal, following the below steps:
1. Set your e-mail spam filter to accept e-mail from procureport.com and procureportmail.com.
2. At https://nycers.procureport.com/ click the magnifying glass icon for this RFP under “Posted
Sourcing Events”. On the page that appears click “Request Access” and follow the prompts.
3. You should receive an e-mail notification within one business day regarding access. If you do not
receive an e-mail, promptly e-mail the NYCERS designated contact (Section G.1).
4. Follow the instructions in the e-mail notification to gain access to the RFP materials.

F.1. Technical Proposal

The technical proposal must include all of the following fully completed templates. Note that some
templates require the Respondent to furnish certain specific attachments to accompany the completed
templates (e.g., financial report):
• Template A. Customer History
• Template B. Respondent Firm Customer References
• Template C. Technical Proposal
− Must also include the complete set of attachments specified in Template C.

RFP ID # 04282020-LRP Page 26

 Software Quality Assurance and Validation Services for the LRP

F.2. Cost Proposal

The cost proposal must be a firm fixed price for the delivery of all RFP requirements and the Respondent’s
technical proposal (e.g., services), and must be submitted using Template D. The proposed price must be
adequate to accomplish the entire project scope and meet all RFP terms and requirements, with all
services components aligning to the Respondent’s project plan. NYCERS may require Respondents, at any
time, to detail their cost proposal basis and provide evidence that the proposed firm fixed price is
reasonable for a project of this nature. Respondents will not be entitled to additional monies, special
considerations or contract changes as a result of any form of underestimating or underpricing of any portion
of the project scope or other costs necessary to meet all RFP terms and requirements. NYCERS reserves
the right, at its sole discretion, to reduce any Respondent’s evaluation score due to any cost proposal
concerns.
Furthermore, the entirety of each Respondent’s cost proposal will be the requested information populated
on Template D. Respondents may not include any supplemental or unsolicited information with their cost
proposal. Including any such information with the cost proposal may result in a non-responsive
determination.

F.3. RFP Exceptions

The vendor will be contractually bound by all aspects of this entire RFP package, without negotiation,
except where RFP exceptions were provided with the Respondent’s proposal using Template E. RFP
exceptions are any specific aspects or components of the entire RFP package, including attachments,
addendums, ancillary documents, any aspects of the RFP marked “sample” or “example,” etc., which a
Respondent will not agree to be contractually bound to with NYCERS, as published, as a result of this
procurement.
All Respondents must submit a completed Template E with their proposal that clearly and specifically
describes all exceptions with the RFP as defined above or alternatively states they have no exceptions.
NYCERS will not entertain RFP exceptions made after proposals are received, and only those exceptions
identified in Template E will be considered by NYCERS. Any exceptions found elsewhere in the proposal
will be superseded by the provisions of this RFP and will not be considered or honored by NYCERS and
may lead to a non-responsive determination. Failure to cite exceptions constitutes full acceptance of the
RFP elements, regardless of whether the Respondent’s omission was unintentional. In addition, RFP
exceptions are not an opportunity for Respondents to communicate preferences for alternatives for any
given aspect of the RFP package where the Respondent would ultimately accept the original published
content.
All RFP exceptions provided must meet the following requirements:
• Each exception must be standalone and not contingent or dependent on any other exception in any
way.
• For each exception an alternative must be proposed that would be acceptable to the Respondent
and that the Respondent reasonably believes will be acceptable to NYCERS.
− NYCERS reserves the right to accept any exception alternative proposed without further
negotiation or any opportunities for revision.

RFP ID # 04282020-LRP Page 27

 Software Quality Assurance and Validation Services for the LRP

• All information provided regarding exceptions must be specific and precise. Generalizations,
deferment, “hedging” and similar language are not permitted.
NYCERS will review each exception and make one of the following general determinations at its sole
discretion:
• To accept the exception and proposed alternatives as submitted by the Respondent.
• To reject the exception and proposed alternatives as submitted by the Respondent as
unacceptable to NYCERS and deem the proposal non-responsive.
• To opt to discuss or negotiate the exception and proposed alternatives with the Respondent at a
future stage of the procurement process (e.g., contract negotiations).
− NYCERS will not accept as a result of contract negotiation any terms less favorable to
NYCERS than the proposed alternatives submitted by the Respondent in their written
proposal, as solely determined by NYCERS.
NYCERS may choose, at any time, and at its sole discretion, to change its determination on any RFP
exception. For example, for any exception NYCERS is willing to negotiate, cease negotiations and either
accept an alternative originally in the Respondent’s proposal without modification or negotiation, or to reject
the exception and any proposed alternatives and potentially as a result deem the proposal non-responsive
and end negotiations. A Respondent’s evaluation score may also be impacted as a result of RFP
exceptions as deemed appropriate by NYCERS.

F.4. Additional Proposal Templates

NYCERS requires the following additional templates to be completed and received simultaneously with
Respondent proposals:
• Template F. Vendor Information Form
• Template G. Vendor Acknowledgement of NY State Freedom of Information Law
• Template H. Acknowledgement of Addenda
• Template I. Bidder’s Certification of Compliance with Iran Divestment Act
• Template J Doing Business Data Form
• Template K. Affirmations and Broker’s Forms
• Template L. Responsiveness Certification Form

F.5. Insurance Policies

In addition to the Certificates of Insurance and Certification of Insurance Broker or Agent (Template K),
Respondents shall provide copies of insurance policies for the following insurance types listed in Schedule
A of City Appendix A: 1) Workers’ Compensation, 2) Disability Benefits Insurance, 3) Employers’ Liability,
4) Commercial General Liability, 5) Professional Liability/Errors & Omissions, 6) Crime Insurance, and 7)
Cyber Liability Insurance. Respondents must use the Insurance Policy Checklist, included in Template K,

RFP ID # 04282020-LRP Page 28

 Software Quality Assurance and Validation Services for the LRP

to indicate which insurance policies are being submitted and any types of insurance the Respondent does
not hold. If any provided insurance policies are deemed by the Respondent to be confidential, Respondent
must so indicate in Template G, Vendor Acknowledgement of NY State Freedom of Information Law.

RFP ID # 04282020-LRP Page 29

 Software Quality Assurance and Validation Services for the LRP

G. Procurement Process and Requirements

One requirement for Respondents to be eligible to be considered responsive (see Section A.2), is meeting
all procurement process requirements (e.g., deadlines) and participating in all aspects of the procurement
process as described in this RFP. Any Respondent who does not meet procurement requirements or
adhere in good faith to the prescribed procurement process may be deemed non-responsive. Note that
NYCERS expects to narrow down the field of Respondents as it moves through the procurement process
described in this section and may not invite all Respondents to participate in later stages of the process
such as respondent presentations and contract negotiations. NYCERS also reserves the right to modify or
cancel the procurement process at any time, for any reason.

G.1. Designated Contact

All inquiries to NYCERS regarding this procurement, or anything potentially related to this procurement, are
only permitted to be made in writing via e-mail to the designated NYCERS contact below. Inquiries to
NYCERS made outside of the designated contact may disqualify the Respondent and any other related
parties from the procurement at NYCERS sole discretion.
Contact: Tenaye Wolde
E-Mail: [email protected]

G.2. Procurement Milestones and Anticipated Timings

Table 12 summarizes procurement milestones and anticipated timings. Respondents are required to
adhere to the stated timings and deadlines unless amended by NYCERS. NYCERS does not guarantee the
milestones and timings presented will remain unchanged and may amend the milestones and timings at its
sole discretion at any time. Additional information regarding the milestones listed in Table 12 are included
in this section. All question submissions and correspondence are to be made via e-mail to the designated
contact (see Section G.1) unless otherwise specified.
All other RFP related Appendices, Forms, and Templates must be accessed and submitted through the
NYCERS Procurement Portal via this link: https://nycers.procureport.com/
Where a specific NYCERS prescribed template is required, any partially completed or modified templates
received by NYCERS (e.g., legal language changes), and/or that include any additional unrequested
documentation (e.g., riders), will not be accepted.
Table 12. Procurement Milestones

Milestone Description Timeframe or Deadline

RFP Published in City Record Online March 13, 2020
Written Respondent Questions Due Must be received by NYCERS no later than
March 24, 2020 5:00 PM EST
NYCERS’ Response to Questions published in City Record March 31, 2020 (estimated)
Online
Respondent Electronic and Hard Copy Proposal Submission April 28, 2020 5:00 PM EST
Deadline

RFP ID # 04282020-LRP Page 30

 Software Quality Assurance and Validation Services for the LRP

Milestone Description Timeframe or Deadline

Proposal Responsiveness Review Period April – May 2020 (estimated)
Initial Evaluation Committee Review of Written Proposals May – June 2020 (estimated)
Respondent Presentations and Demonstrations
Note: NYCERS may not invite all Respondents to participate
in Presentations and Demonstrations
Notice of Award and Project Start January 2021 (estimated)

G.3. Written Respondent Questions

NYCERS designated contact must receive from all Respondents their written questions by the deadline
specified in Table 12. Even if a Respondent has no specific questions, Respondents must send an e-mail
to the designated contact to that effect that must be received by the deadline specified in Table 12.
Respondents are hereby notified this is the only fully open opportunity to ask material questions or request
clarifications regarding the RFP.

G.4. Responses to Written Vendor Questions

Responses to all written questions received will be made available to all Respondents electronically on The
City Record Online.

G.5. Receipt of Respondents’ Electronic and Hard Copy Proposals

Proposals must be received by NYCERS by the deadline in Table 12 in two formats as described below.
Proposals received after the deadline will be deemed non-responsive. However, Respondents will be
afforded an opportunity to file an appeal pursuant to the NYC Procurement Policy Board (PPB) Rules.
Pursuant to New York City’s Procurement Policy Board Rules, Vendors have the right to appeal NYCERS non-
responsiveness determinations within five (5) days and NYCERS non-responsibility determinations within ten
(10) days and to protest NYCERS’ determination regarding the solicitation or award of a contract in writing to
NYCERS, 335 Adams Street, Suite 2300, Brooklyn, NY 11201.
The determination of appeals by NYCERS will be made in writing and are final. Any administrative RFP
process questions about proposal submission should be submitted to the designated contact as soon as
possible (such questions are not subject to the written questions process described in Section G.3).
• Electronic copy – an electronic copy of the proposal, including all templates and attachments,
must be uploaded to NYCERS procurement portal: https://nycers.procureport.com/.
− Instructions on gaining procurement portal access are provided in Section F.
− Respondents are expected to take the time to closely follow the instructions within this RFP
and as shown on the NYCERS procurement portal or the Respondent’s proposal may be
deemed non-responsive.
− Respondents are solely responsible for allowing adequate time to properly upload their
proposal before the submission deadline in Table 12, including ensuring sufficient time to
respond to unforeseen circumstances, such as an interruption in the Respondent’s internet
connection that prevents or degrades access to the NYCERS procurement portal.

RFP ID # 04282020-LRP Page 31

 Software Quality Assurance and Validation Services for the LRP

• Hard copies – eight (8) exact ring-bound paper copies of the electronic proposal, including all
templates and attachments, must be received by NYCERS no later than the proposal submission
deadline in Table 12.
− Cost proposal copies must be provided in a separate sealed envelope.
− For any hard copies sent via mail or carrier, Respondents, not the postal service or selected
carrier, are solely responsible for ensuring adequate time to help ensure NYCERS receives the
hard copy proposals in advance of the submission deadline in Table 12.
o Respondents are strongly encouraged to purchase appropriate proof of delivery services.
− Hard copies may alternatively be delivered in person by the Respondent firm. Respondents
are solely responsible for ensuring adequate time to help ensure NYCERS receives the hard
copy proposals in advance of the submission deadline in Table 12, including, but not limited to,
accounting for potential travel delays and allocating time to go through building security.
− All hard copies are to be received at NYCERS executive offices (below). In person deliveries
by carriers or the Respondent must check-in at NYCERS reception on the 22nd floor.
New York City Employees’ Retirement System
Attn: Tenaye Wolde
335 Adams Street, Suite 2300
Brooklyn, NY 11201-3724
The New York City Comptroller is charged with the audit of the contract in New York City. Any Vendor who
believes that there has been unfairness, favoritism or impropriety in the proposal process should inform the
Comptroller, Office of Contract Administration, 1 Centre Street, Room 1005, New York, NY 10007; telephone
number (212) 669-2323.

G.6. RFP Responsiveness and Initial Review of Written Proposals

For all proposals received, NYCERS will first conduct a review to determine if the proposal is responsive
(see Section A.2). Responsiveness will solely be determined by NYCERS. Proposals deemed non-
responsive will not be further considered. Should a proposal be deemed non-responsive, Respondent will
be afforded an opportunity to file an appeal pursuant to the NYC Procurement Police Board (PPB) Rules.
The determination of appeals may by NYCERS will be made in writing and will be final. Only proposals
deemed responsive by NYCERS will be evaluated (per the criteria set forth in Section I). Please note that
NYCERS reserves the right to award the contract on the basis of the initial proposals received, without
soliciting a best and final offer (BAFO), further discussions or negotiations with the Respondent. Therefore,
it is important each Respondent’s initial proposal should contain its best terms.

G.7. Respondent Presentations and Demonstrations

After the initial review of proposals, NYCERS anticipates inviting some, but not all, Respondents to deliver
an in-person presentation to NYCERS as well as to participate in demonstration exercises. In order to be
considered for selection, Respondents invited must accept NYCERS’ invitation and in good faith fully
participate in this stage of the procurement process as directed by NYCERS. Regarding in-person
presentations, NYCERS anticipates issuing guidelines Respondents must adhere to (e.g., timing

RFP ID # 04282020-LRP Page 32

 Software Quality Assurance and Validation Services for the LRP

limitations) in advance of scheduled presentations. Demonstration exercises are anticipated to be some

combination of delivery team interactions with NYCERS staff as well as demonstrating how proposed
testing tools would be used during project implementation. NYCERS plans to provide specific details when
distributing invitations to Respondents.

G.8. Contract Negotiations

If NYCERS chooses to negotiate with a Respondent, in order to be considered for selection, the
Respondent invited must accept NYCERS’ invitation and fully participate in this stage of the procurement
process as directed by NYCERS. The scope and timeframe of contract negotiations will be solely
determined by NYCERS. NYCERS is under no obligation to move forward with any Respondent(s) it
chooses to negotiate with.

G.9. Final Evaluations

NYCERS will finalize evaluations and subsequently determine if any contracts will be awarded. NYCERS is
under no obligation to award any contracts at the end of the procurement process. Should NYCERS decide
not to contract with the Vendor deemed most responsible and responsive, it in its sole discretion may
negotiate with the next Vendor deemed most responsible and responsive.

RFP ID # 04282020-LRP Page 33

 Software Quality Assurance and Validation Services for the LRP

H. Terms and Conditions

Submission of a response or a proposal shall constitute the Respondent’s acknowledgement it has fully
reviewed the entire RFP package, and fully accepts the terms and conditions herein, including Appendix A
and Appendix A1 barring any the specific exceptions provided by the Respondent with its proposal (see
Section F.3).

H.1. Multi-Term Award

NYCERS intends to award this as a multi-term contract (more than one year). The multi-term contract is
subject to modification or cancellation if adequate funds are not appropriated to NYCERS to support
continuation of performance in any fiscal year succeeding the first. The contract is subject to modification
or cancellation if the selected Respondent’s performance is not satisfactory. The NYCERS Contracting
Officer shall notify the selected Respondent as soon as is practicable if the funds are, or are not, available
for the continuation of the multi-term contract for each succeeding fiscal year.

H.2. NYCERS Project Staffing

NYCERS does not guarantee assignment of personnel or other resources, including both individuals
directly employed by NYCERS and any contractors NYCERS may employ outside of the selected
Respondent, to the project, and will not enter into a binding commitment with the Respondent regarding
NYCERS assigned resources (e.g., time, number, skill, experience). Any formal or informal information on
NYCERS staff resources and commitment level to the project shall in all instances be considered estimates
and shall not be interpreted under any circumstances as a binding commitment by NYCERS, including
when such information is provided in contract documentation. NYCERS reserves the right to change,
reduce, or increase NYCERS assigned resources at any time, for any reason, and without advance notice
to the Respondent, before and during the project. Respondents are exclusively responsible for delivery of
the project within the agreed upon timetable, including where delivery is being provided by the
subcontractors of the selected Respondent. As such, every Respondent’s proposed staffing plan must be
designed with the assumption that NYCERS resources may not be available as the selected Respondent
expects or planned for. The selected Respondent will not be entitled to special considerations (e.g.,
contract modifications, deadline adjustments, additional monies) due to NYCERS project staffing.

H.3. Staff Replacements

NYCERS, at its sole discretion, reserves the right to remove any of the selected Respondent’s staff or
subcontractor’s staff from the project, for any reason and without advance notice. Upon written notification,
the selected Respondent shall have no more than ten (10) business days to furnish NYCERS with resumes
for a minimum of three (3) replacement candidates with equal or greater skills and experience as the
removed staff member, and two (2) references for each proposed replacement candidate. NYCERS may
require an in-person interview for any or all proposed replacement candidates, and the selected
Respondent will ensure that the replacement candidate(s) can be onsite for an interview within ten (10)
business days of NYCERS’ request. The selected Respondent shall also ensure that the replacement
candidate is available to start within 15 business days of NYCERS’ candidate selection. In the event none
of the initial proposed replacement candidates are acceptable to NYCERS, NYCERS reserves the right to
require the selected Respondent to furnish additional resumes indefinitely until an acceptable candidate is
selected.

RFP ID # 04282020-LRP Page 34

 Software Quality Assurance and Validation Services for the LRP

Unless mutually agreed upon by both parties, the selected Respondent and its subcontractors must
maintain composition of the project team at the agreed upon levels throughout the project timeline.
Respondent must receive express written consent from NYCERS before removing staff members from the
project, reducing staff member involvement, reassigning staff members, replacing staff members, or taking
any similar staffing actions. Exceptions will only be considered for staffing situations outside the control of
the selected Respondent (e.g., staff member resigns from the selected Respondent’s firm). In these
events, NYCERS must be notified within three (3) days of such event out of Respondent’s control, and the
staffing replacement process and requirements described above will apply. NYCERS may require proof of
such situations from the selected Respondent.
NYCERS shall be entitled to seek damages if any of the selected Respondent’s staffing losses and/or
replacements contribute to project delay or degradation of quality work products.

H.4. Warranty
Warranty will begin upon the first end-user solution deployment and will end 12 months from NYCERS’ final
acceptance of all project deliverables after full implementation. Defects discovered near the end of the
warranty period shall automatically extend the warranty period by 90 days past the date the last defect was
resolved unless otherwise agreed by NYCERS. Similar automatic extensions shall apply to defects
discovered during the extended warranty period. All automatic extensions shall be provided at no
additional cost to NYCERS. NYCERS shall notify Respondent in writing of the automatic extension.
Notwithstanding the foregoing, Respondent is responsible for the extension period and a failure to provide
notification does not constitute a waiver. In addition, NYCERS shall be entitled to purchase from the
vendor an extended warranty for any number of additional months up to three (3) years, commencing on
the day the original warranty ends (after any automatic extensions), for identical terms, coverage, and
levels of service as the original warranty (e.g., including the terms that will automatically extend the
warranty beyond its original term based on a qualifying event). In the event NYCERS does not purchase
the maximum possible extension, NYCERS reserves the right to purchase further extensions, at the original
agreed upon price, up to the maximum at a later time.
All proposed warranties must be managed directly by the vendor, and not by a subcontractor, and must
cover the entirety of the solution, including customizations. The warranty shall not be voided, in part or in
whole, by any supported changes to the solution made by NYCERS’ staff post-deployment (e.g., defects
discovered after actions such as modifying business rules or workflow, base product upgrades, enabling
and configuring new features or other capabilities supported by the product), and such post-deployment
changes shall be fully covered by the warranty. The warranty must also meet all SLAs as described in
Appendix 5.

H.5. Cost to Propose and/or Negotiate

All costs directly or indirectly incurred in reviewing, responding to, and or participating in this RFP, shall be
solely borne by the Respondent. NYCERS shall not be liable for any costs or expenses incurred by
Respondents as a result of this RFP, the RFP procurement process, NYCERS’ modification or withdrawal
of this RFP, NYCERS’ determination that a contract shall not be awarded, and/or any costs associated with
the negotiation of the Agreement.

RFP ID # 04282020-LRP Page 35

 Software Quality Assurance and Validation Services for the LRP

H.6. Collusion or Fraud

By submitting a proposal, each Respondent represents and warrants that its proposal is not made in
connection with any other Respondent that is submitting a response to this RFP, and that its proposal is in
all respects fair and without collusion or fraud. Any evidence of agreement or collusion among
Respondents to illegally restrain freedom from competition, such as by agreement to offer a fixed price or
otherwise, will disqualify all involved entities (e.g. Respondent, subcontractor) from this procurement and
any related procurements.

H.7. Rights to Modify or Withdraw RFP

NYCERS reserves the right to modify or withdraw this RFP at any time. Any notice or amendments by
NYCERS to this RFP shall be set forth in writing, and sent to all prospective Vendors known to have
received this RFP and/or published on the City Record online. Such amendments shall be distributed
within a reasonable time to allow prospective Vendors to consider them in preparing their proposals.

H.8. Travel and Other Expenses

Respondents shall build in travel time, travel costs, and any other appropriate expenses within its Cost
Proposal. NYCERS shall not be liable for any additional monies resulting from an underestimation of, or
unanticipated travel and other expenses for the entire duration of the project. Unless otherwise specified
by NYCERS, each travel expense will not be independently billed. Respondents are not permitted to
directly or indirectly attribute time or costs for expenses to the project, such as: travel time, product or job
function training, time to complete administrative items necessary to meet the selected Respondent’s
internal firm requirements.

H.9. Respondent Pricing

Respondents are hereby notified that, unless the proposal is withdrawn, the prices stated in their proposals
and BAFO’s are final and valid until contract award. Offers may be withdrawn in writing to NYCERS before
an award is made. NYCERS reserves the right to request a cost breakdown of the Respondent’s proposed
price, including details above and beyond what is required in the Cost Proposal template (Template E).
Respondents are not permitted to apply a direct or indirect markup of any kind for any third-party products
or services proposed that may ultimately be procured by NYCERS through contracts with the selected
Respondent, including but not limited to third-party software licenses and maintenance costs. NYCERS
reserves the right directly procure third-party software licenses and maintenance services.

H.10. NYCERS Prescribed Tools and Software

NYCERS may request the selected Respondent use specific software or tools to support project
implementation, or may change the tools or software used to support the project at any time during the
course of the project. The selected Respondent will comply with such requests at no additional cost to
NYCERS and without causing negative impacts to the project (e.g., delays).

H.11. Respondent Proprietary Tools and Software

Where the selected Respondent is permitted by NYCERS to use proprietary tools or software, the selected
Respondent agrees to provide full access and transparency into such tools or software to the extent
requested by NYCERS.

RFP ID # 04282020-LRP Page 36

 Software Quality Assurance and Validation Services for the LRP

H.12. Ownership of Work Products and Intellectual Property

NYCERS will have all ownership rights to hardware and software, or modifications thereof, as well as
associated documentation designed, developed, or installed. All data, technical information, materials
gathered, originated, developed, prepared, used, or obtained in the performance of the contract, including,
but not limited to, all reports, surveys, plans, charts, literature, brochures, mailings, recordings (video and/or
audio), pictures, drawings, analyses, graphic representations, software computer programs and
accompanying documentation and print-outs, notes and memoranda, written procedures and documents,
regardless of the state of completion, that are prepared for or are a result of the services required under
this contract shall be and remain the property of NYCERS and shall be delivered to NYCERS upon 30
days’ notice by NYCERS. With respect to software computer programs and/or source codes developed for
NYCERS, the work shall be considered “work for hire,” i.e., NYCERS, not the Respondent or subcontractor,
shall have full and complete ownership of all software computer programs and/or source codes developed.
All work products and deliverables produced under contracts awarded as a result of this proposal will be
the exclusive property of NYCERS. This includes, but is not limited to, software, test scripts (manual,
automated), documentation, and development materials. Respondents shall not sell a work product or
deliverable produced under a contract awarded as a result of proposals without explicit permission from
NYCERS.
NYCERS’ ownership interest in both the solution and the source code delivered, for the base components
of the solution, the customized elements of the solution, and the enhancements provided to each through
post-implementation, out year support, is a license to use them in perpetuity.
The Respondent’s proposal must clearly describe the terms of all licensing considerations, such as an End
User License Agreement.

H.13. Source Code Escrow/ Software Requirements

The vendor’s proposed solution must NOT include software or hardware locks, traps, dongle keys, or
similar security measures that would in any way deny NYCERS full and complete access.
NYCERS requires a nonexclusive, unlimited perpetual license to software for source code, as well as
associated documentation designed, developed, or installed. Such nonexclusive license shall apply to all
data, technical information, materials gathered, originated, developed, prepared, used, or obtained in the
performance of activities associated with the procurement. Additionally, the scope of this license shall
include, but not be limited to, all reports, surveys, plans, charts, literature, brochures, mailings, recordings
(video and/or audio), pictures, drawings, analyses, graphic representations, software computer programs
and accompanying documentation and print-outs, notes and memoranda, written procedures and
documents, regardless of the state of completion, that are prepared for or are a result of the services
required and to be delivered under this procurement.
The source code for any software (be it the customized line-of-business application, middleware, a code
generator, a specialized I/O routine, or any similar or related item) which is developed by the Respondent,
or an affiliate company that is 20% or more owned by the Respondent, and used in the new system must
be delivered to NYCERS. The Respondent must agree to deliver application source code with the delivery
of each functional rollout phase during the project. NYCERS’ acceptance of a phase will be contingent
upon this requirement being met. In addition, once the first functional rollout phase has been delivered,

RFP ID # 04282020-LRP Page 37

 Software Quality Assurance and Validation Services for the LRP

source code updates must be delivered no less frequently than quarterly. Source code for linkages to/from
the system (which may be developed by a ‘sub-contractor) is similarly to be provided. It is to be updated
and delivered to NYCERS throughout the duration of the project.
In addition, the source code for any third party software that is delivered and/or licensed to NYCERS as
part of the new solution must be escrowed on NYCERS’ behalf if, at the time of final turnover and
acceptance of the new system to NYCERS, that software product has ever previously been escrowed for
the benefit of any other client of the third party provider.

H.14. Agreement
In the event that NYCERS chooses to proceed with a Respondent, the proposed Agreement that NYCERS
anticipates to award will be based on the proposal submitted by the chosen Respondent and specifications
of this RFP. The proposed Agreement between NYCERS and the vendor shall be a combination of the
specifications, terms, and conditions of this RFP, including the terms included in any documents attached
to the RFP, the Respondent’s technical and cost proposals, any written clarifications or changes made in
accordance with the provisions set forth within this RFP, and any other terms deemed necessary by
NYCERS or negotiated between the parties. NYCERS reserves the right to terminate all contracts for any
reason at any time without penalty. By submitting a proposal, each vendor acknowledges its acceptance of
these specifications, terms and conditions without change except as otherwise specified as RFP exceptions
(see Section F.3).

H.15. Change Requests

Change requests will be exclusively managed via the change control provisions established in the
agreement with the selected Respondent. NYCERS may, at its sole discretion, require the selected
Respondent to provide information, documentation and/or evidence as a condition of considering a change
request. For example, evidence that the selected Respondent provided NYCERS with advance notice of a
potential change request and explored reasonable alternatives to a change request may be required. The
selected Respondent will not be entitled to additional monies related to change requests (e.g., preparing,
researching). Moreover, the selected Respondent shall not recover retroactively for costs incurred prior to
the contractual execution of a NYCERS approved change request. All proposed cost increases associated
with proposed change requests will require detailed information as determined by NYCERS, which at
minimum shall include a rationale as to why the cost increase is permissible to be sought per the contract,
an accounting of all actions taken to avoid the need for a cost increase, and a detailed buildup of the
proposed cost increase.

H.16. Assumptions
To help ensure all Respondents’ proposals are based on the same set of RFP details and other
foundational criteria, Respondents are not permitted to make their own assumptions in their technical and
cost proposal, or with regard to any activity related to their proposal or this procurement (e.g., oral
presentations, contract negotiations). By submitting a proposal, Respondents certify they have requested
clarification from NYCERS on all elements of the RFP during the written question period (see Section G.3)
and have updated their entire proposal submission (e.g., technical, cost) accordingly based on any
NYCERS’ responses. Any assumptions found in proposals will be disregarded by NYCERS without an
opportunity for the Respondent to revise their proposal. Respondents are reminded of the option to submit

RFP ID # 04282020-LRP Page 38

 Software Quality Assurance and Validation Services for the LRP

exceptions to this RFP as they see fit (see Section F.3). Furthermore, Respondents hereby agree that
assumptions, or any equivalent, will not be added or otherwise included in any subsequent contract
documentation that may be agreed to with NYCERS.

H.17. Subcontractors
NYCERS will hold the prime contractor solely responsible and liable for all its subcontractors. NYCERS
requires full visibility into all firms involved in the project. Therefore, the selected Respondent may only
employ subcontractors with the express written approval of NYCERS. Similarly, subcontractors, at any
level, may only employ subcontractors with the express written approval of NYCERS. The selected
Respondent is solely and fully responsible and accountable for all work performed by subcontractors,
including full costs of remediating any inadequate work products; including the event that NYCERS, for any
reason, requires subcontractors be removed from the project. The selected Respondent will warrant that
subcontractors comply with all terms and conditions in this RFP and any resulting Agreements. The
selected Respondent’s formal agreements and any informal arrangements with subcontractors must:
• Not place any restrictions or limitations on subcontractors from contracting separately with
NYCERS at any time for any reason.
• Permit subcontractors to interact directly with NYCERS, including outside the presence of any
selected Respondent staff.
• Require subcontractors to be fully transparent with NYCERS regarding their activities, work product
and any other project elements.
• Not penalize or restrict in any way subcontractors from directly communicating to NYCERS project
concerns, risk or other information material to the success of the project, including any situations
that, in the selected Respondent’s opinion, may not create a favorable perception of the selected
Respondent to NYCERS.

H.18. Procurement Flexibility

NYCERS, at its sole discretion, may purchase any components/elements and/or services (e.g., hardware,
software, cloud services) for this project by means of any procurement vehicle(s) of its choosing, and is not
obligated to purchase any components/elements and/or services from the selected Respondent. NYCERS
shall not owe the selected Respondent monies or other compensation for exercising its discretion to make
such purchases outside the contract with the selected Respondent.

H.19. Termination following the Project Assessment Phase

Respondents are advised that NYCERS expects to complete the effort with a single Respondent. However,
the proposals and ensuing Agreements shall provide that at the end of the Project Assessment Phase (i.e.,
Phase 0), NYCERS may, at its sole discretion and with no penalty, terminate the effort at any time including
not moving forward with the Respondent to a subsequent phase. NYCERS liability will be limited to
payment of an amount equal or less than the costs set forth in the payment structure set up based on
deliverables and accepted by NYCERS. NYCERS shall not be liable for any other cost, overhead, profits,
or damages.
This section does not apply to termination in the event of a material breach of contract.

RFP ID # 04282020-LRP Page 39

 Software Quality Assurance and Validation Services for the LRP

Should NYCERS elect to terminate the project at the end of the Project Assessment Phase, the termination
shall be subject to the following terms and conditions:
• Termination shall become effective upon NYCERS’ sending a written or electronic notice to the
Respondent.
• Termination shall not be considered for cause or for default, and notice of termination shall not
constitute an admission or accusation of any wrongdoing on the Respondent’s part in connection
with the termination.
• Upon termination, NYCERS and the Respondent shall each execute a covenant not to sue and a
general release of all claims or potential claims, including those for costs, profits, liabilities, or
damages, as of the date of termination. However, NYCERS may assert a claim that Respondent
failed to return NYCERS materials and data, which shall include all original files and other
materials, and all copies, including unauthorized copies, of such materials in all media, including
portions of either originals or copies, in Respondent’s possession or under its control. Respondent
shall agree that it will refrain from using any such materials or copies thereof. NYCERS’ materials
consist of but are not limited to devices, records, data, notes, reports, proposals, business cards,
letters, specifications, drawings, equipment, and other materials. Respondent shall agree to erase
or destroy any NYCERS materials and portions thereof contained in all types of computer memory,
and so warrant in writing to NYCERS within thirty (30) days of termination.

H.20. Termination without Cause

Termination shall become effective upon NYCERS’ sending a written or electronic notice to the
Respondent. NYCERS shall only be liable for payment of an amount equal to the costs allocated to the
phase that was completed and accepted by NYCERS; NYCERS shall not be liable for any other cost,
overhead, profits, or damages.
Termination shall not be considered for cause or for default, and notice of termination shall not constitute
an admission or accusation of any wrongdoing on the Respondent’s part in connection with the termination.
Upon termination, NYCERS and the Respondent shall each execute a covenant not to sue and a general
release of all claims or potential claims, including those for costs, profits, liabilities, or damages, as of the
date of termination. However, NYCERS may assert a claim that Respondent failed to return NYCERS
materials and data, which shall include all original files and other materials, and all copies, including
unauthorized copies, of such materials in all media, including portions of either originals or copies, in
Respondent’s possession or under its control. Respondent shall agree that it will refrain from using any
such materials or copies thereof. NYCERS’ materials consist of but are not limited to devices, records,
data, notes, reports, proposals, business cards, letters, specifications, drawings, equipment, and other
materials. Respondent shall agree to erase or destroy any NYCERS materials and portions thereof
contained in all types of computer memory, and so warrant in writing to NYCERS within thirty (30) days of
termination.

H.21. Permissible Work Locations

The selected Respondent shall be available to meet, conference, or video chat with NYCERS during all
business hours regardless of the selected Respondent’s location.

RFP ID # 04282020-LRP Page 40

 Software Quality Assurance and Validation Services for the LRP

For all US-based resources, the selected Respondent shall take all necessary and appropriate steps to
determine that all personnel are legally eligible to work in the United States, and that such eligibility is
maintained at all times during which any such personnel is performing pursuant to the proposed solution.
Any work that NYCERS agrees may be performed off-site or off-shore, including cloud-based services,
NYCERS may, in its discretion, at any time, during ordinary business hours, inspect the site or site(s) of the
Contractor or any subcontractor engaged in contract performance. The Contractor shall guarantee
NYCERS appropriate access to the site to perform such inspections.
Contractor must satisfy and comply with all NYCERS security requirements and may be subject to
additional confidentiality requirements applicable to any such off-site and off-shore resources that NYCERS
agrees to. In no event will work performed by resources off-site and/or off-shore be priced based upon an
hourly rate greater than the rates applicable to on-site resources.

H.22. Firm Fixed Price for Complete Delivery of Scope and Requirements
Cost for all services shall be a firm fixed price for the complete delivery of all scope and requirements
regardless of the time and effort required by the selected Respondent and any subcontractors, including in
cases where time and effort exceeds the selected Respondents original plans and estimates. Firm fixed
price shall under no circumstances be interpreted as anything else. For instance, if an agile methodology is
being proposed, a firm fixed price shall not mean a firm fixed price for a set number of development
“sprints” regardless of the scope completed and/or requirements met at the end of the planned set of
sprints. Or, for instance, if any given requirement is found by the selected Respondent to require more time
and effort than originally planned, such as it was anticipated by the Respondent to be fulfilled “out-of-the-
box” and then subsequently it is determined by NYCERS, the Respondent, or both, that the requirement
can only be fulfilled through customization, the Respondent shall not be entitled to additional compensation
(direct or indirect) or special considerations. Respondent agrees their proposed firm fixed price is
reasonable and realistic and is not in any way a “minimum” or “starting” price. Respondent will not be
entitled to additional compensation (direct or indirect or special considerations due to underestimating or
underpricing the firm fixed price. Furthermore, by submitting a proposal, Respondent certifies it has
requested clarification from NYCERS on all elements of the RFP during the written questions period (see
Section H.5) necessary to furnish a compliant firm fixed price as described herein and have also updated
their proposed firm fixed price and entire proposal submission (e.g., technical, cost) accordingly based on
any NYCERS’ responses.

H.23. Required Deliverables

Appendix 1 lists the deliverables required to be furnished by the vendor to NYCERS throughout the project.

H.24. Deliverable Expectations Document

For each deliverable, the vendor will be required to provide a deliverable expectation document (DED) on a
format NYCERS prescribes (see Appendix 2). Work performed on deliverables without a NYCERS-
accepted DED is at the Contractor’s risk.

RFP ID # 04282020-LRP Page 41

 Software Quality Assurance and Validation Services for the LRP

H.25. Final Deliverable Review

A deliverable may be submitted for final review when the Contractor, in good faith, believes all
requirements for that deliverable have been met, after all NYCERS feedback on any required draft
deliverables has been addressed (see Appendix 2), and if in alignment with agreed upon timings. For
deliverables NYCERS solely determines are eligible for final deliverable review, NYCERS shall have no
less than fifteen (15) full business days to either accept the deliverable or provide written deliverable
feedback to the Contractor, which the vendor shall be required to address completely and comprehensively
prior to resubmission.

H.26. Payments
Progress payments against the total cost of the phase or release will only be made when the vendor
achieves a set of NYCERS-prescribed payment milestones as described in this section. Progress
payments will not be made on a per deliverable basis. Any amounts retained will be released upon
NYCERS full acceptance of all work and deliverables at the completion of the entire project. Contractor is
responsible for preparing and furnishing invoices to NYCERS prior to any payments being made and such
payments will be made consistent with NYCERS 30 days terms. All payments must be received from
NYCERS via an electronic fund transfer (EFT).

Project Assessment Phase

Only one payment will be made for the Project Assessment Phase (i.e., Phase 0). The Project Assessment
Phase will be payable upon NYCERS full acceptance that the Phase has been completed successfully and
NYCERS’ formal acceptance of all Project Assessment Phase deliverables identified in Appendix 1.
NYCERS will pay no more than $350,000 for the Project Assessment Phase. Any costs above this amount
shall be the sole responsibility of the vendor and at no point will be payable. A 10% retainage shall apply to
the Project Assessment Phase.

Phased Solution Releases

Each release will have the following non-negotiable payment milestones. A payment milestone will be
considered achieved if all associated deliverables and realized benefits as assigned to the milestone, as
well as any updates to previously accepted deliverables to ensure they are current, are fully accepted by
NYCERS (see Appendix 1). In addition, the Contractor must have successfully achieved all prior payment
milestones leading up to the current payment milestone.
Payment Milestone % of Release Cost Paid
A 10%

B 10%

C 25%

D 45%

Retainage 10%

RFP ID # 04282020-LRP Page 42

 Software Quality Assurance and Validation Services for the LRP

SLA Retainage
An additional retainage of 0.5% for each SLA missed (see Appendix 4) for any reporting interval from the
last payment milestone date to the current payment milestone date shall be withheld from the total payable
amount of the current payment milestone (applies to Project Assessment Phase and all Release phases).
Any SLA retainage withheld during the project will be released with other retainage. NYCERS will solely
determine if an SLA has been missed and there shall be no limiting circumstances attached to SLA
retainage. The selected Respondent is solely and exclusively responsible for achievement of SLAs and
therefore the avoidance of SLA retainage. NYCERS shall in no way be held accountable for the selected
Respondent’s inability to meet SLAs, and therefore the application of SLA retainage as a result, under any
circumstances.

H.27. Off-Shoring
Respondents are permitted to propose use of individuals working off-shore to the extent such proposal
conforms with all parameters, terms and conditions set forth regarding such staffing in this RFP. No more
than one-third (33.333%) of a Respondent’s project resources, as listed in the Respondent’s Cost Proposal
(see Section F.2) may filled by individuals working off-shore. Respondents are advised that NYCERS at its
sole discretion will decide whether off-shore resources will be authorized.

H.28. Vendor’s Acknowledgment of NYS Freedom of Information Law

Vendor must complete and return the Vendor’s Acknowledgment of NYS Freedom of Information Law
(Template G). Vendor shall give specific attention to the identification of those portions of their proposals
the Vendor deems to be confidential, proprietary information or trade secrets and provide any justification
why such materials, upon request, shall not be disclosed by NYCERS. Such information must be easily
separable from the non-confidential section of the proposal.

H.29. RFP Inconsistencies

It is the sole responsibility of Respondents to review all RFP materials in detail and identify any perceived,
potential and/or actual inconsistencies within the RFP and to submit sufficiently detailed written questions
to NYCERS during the designated written Respondent questions period requesting clarification. If
inconsistencies are discovered later in the procurement process or after contract award, NYCERS will
solely determine which version of the inconsistent element prevails. Respondents will be then required to
follow NYCERS determination without any reduction in delivery (e.g., reduced scope, increased timeline)
and without any increase in price to NYCERS.

H.30. City of New York Prompt Payment Policy

Vendors are hereby notified of the City of New York’s Prompt Payment Policy:
(a) The prompt payment provisions of PPB Rule § 4-06 are applicable to payments made under
this Agreement. The provisions generally require the payment to the Contractor of interest on
payments made after the required payment date, as set forth in the PPB Rules.
(b) The Contractor shall submit a proper invoice to receive payment, except where the Agreement
provides that the Contractor will be paid at predetermined intervals without having to submit an
invoice for each scheduled payment.

RFP ID # 04282020-LRP Page 43

 Software Quality Assurance and Validation Services for the LRP

H.31. Iran Divestment Act Compliance Rider for New York City Contractors
The Iran Divestment Act of 2012, effective as of April 12, 2012, is codified at State Finance Law (“SFL”)
§165-a and General Municipal Law (“GML”) §103-g (see Appendix 5). The Iran Divestment Act, with
certain exceptions, prohibits municipalities, including the City, from entering into contracts with persons
engaged in investment activities in the energy sector of Iran. Pursuant to the terms set forth in SFL §165-a
and GML §103-g, a person engages in investment activities in the energy sector of Iran if:
(a) the person provides goods or services of twenty million dollars or more in the energy sector of
Iran, including a person that provides oil or liquefied natural gas tankers, or products used to
construct or maintain pipelines used to transport oil or liquefied natural gas, for the energy
sector of Iran; or
(b) The person is a financial institution that extends twenty million dollars or more in credit to
another person, for forty-five days or more, if that person will use the credit to provide goods or
services in the energy sector in Iran and is identified on a list created pursuant to paragraph (b)
of subdivision three of Section 165-a of the State Finance Law and maintained by the
Commissioner of the Office of General Services.
A bid or proposal shall not be considered for award nor shall any award be made where the bidder or
proposer fails to submit a signed and verified bidder’s certification (Template I).
Each bidder or proposer must certify that it is not on the list of entities engaged in investment activities in
Iran created pursuant to paragraph (b) of subdivision 3 of Section 165-a of the State Finance Law
(Template I). In any case where the bidder or proposer cannot certify that they are not on such list, the
bidder or proposer shall so state and shall furnish with the bid or proposal a signed statement which sets
forth in detail the reasons why such statement cannot be made. The City of New York may award a bid to
a bidder who cannot make the certification on a case by case basis if:
(1) The investment activities in Iran were made before the effective date of this section (i.e., April
12, 2012), the investment activities in Iran have not been expanded or renewed after the
effective date of this section and the person has adopted, publicized and is implementing a
formal plan to cease the investment activities in Iran and to refrain from engaging in any new
investments in Iran: or

H.32. (2) The City makes a determination that the goods or services are necessary for the
City to perform its functions and that, absent such an exemption, the City would be unable
to obtain the goods or services for which the contract is offered. Such determination shall
be made in writing and shall be a public document.Doing Business Data Form Notice
1. Notice
Pursuant to Local Law 34 of 2007, amending the City's Campaign Finance Law, the City is required to
establish a computerized database containing the names of any "person" that has "business dealings with
the city" as such terms are defined in the Local Law. In order for the City to obtain necessary information to
establish the required database, vendors responding to this solicitation are required to complete the
attached Doing Business Data Form (“Data Form”) and return it with this proposal. (If the responding
vendor is a proposed joint venture, the entities that comprise the proposed joint venture must each

RFP ID # 04282020-LRP Page 44

 Software Quality Assurance and Validation Services for the LRP

complete a Data Form.) If the City determines that a vendor has failed to submit a Data Form or has
submitted a Data Form that is not complete, the vendor will be notified by NYCERS and will be given four
(4) calendar days from receipt of notification to cure the specified deficiencies and return a complete Data
Form to NYCERS. Failure to do so will result in a determination that the proposal is nonresponsive.
Receipt of notification is defined as the day notice is e-mailed or faxed (if the vendor has provided an e-mail
address or fax number), or no later than five (5) days from the date of mailing or upon delivery, if delivered.
2. The Data Form
If you have already completed a Data Form (Template J), please submit it as a “No Change,” which you will
find under Section 1. Entity Information. You only need to provide your Entity Name and EIN Number on
the first page of the form and skip to the bottom of the 4th page and complete the certification box. If any of
your information has changed you will need to complete the form as a “Change” under Section 1. Entity
Information and update the sections that have changed. If you are unsure whether you have already
completed the Data Form or when you last completed the Form, please call the Mayor’s Office of Contracts
at (212) 788-8104 or you may send an email with your inquiry to [email protected].

H.33. “Appendix A” General Provisions Governing Contracts for Consultants, Professional,

Technical, Human and Client Services – Notice to Proposers
The City of New York has issued a new Appendix A, General Provisions Governing Contracts for
Consultants, Professional, Technical, Human And Client Services (see Appendix 7).

RFP ID # 04282020-LRP Page 45

 Software Quality Assurance and Validation Services for the LRP

I. Evaluation and Contract Award

This section contains information about how Respondents will be evaluated and contracts, if any, will be
awarded. NYCERS is not obligated to award any contract as a result of this procurement. NYCERS also
reserves the right to award contracts to multiple Respondents, at its sole discretion, as a result of this
procurement.

I.1. Evaluation Procedures

All proposals accepted by NYCERS will be reviewed to determine whether they are responsive or non-
responsive to the requisites of the RFP. Proposals that are determined by NYCERS to be non-responsive
will be rejected and not considered further. NYCERS’ Evaluation Committee will evaluate and score all
remaining proposals as described in this section. Discussions may be conducted with Respondents
submitting acceptable proposals and the panel may request a Best and Final Offer (see Section G).
NYCERS may choose, at its sole discretion, to only engage in such activities with some Respondents but
not others (e.g., Respondents with higher initial evaluation scores) and NYCERS reserves the right to
adjust its initial evaluation accordingly based on the result of such activities throughout the procurement
process (see Section G). However, NYCERS also reserves the right to award the contract on the basis of
the initial proposals received, without discussions. Therefore, each Respondent’s initial proposal should
contain its best terms.

I.2. Evaluation Criteria

To be eligible for evaluation, proposals must be deemed responsive by NYCERS, include all components,
and meet all proposal requirements. Scoring will be done on a weighting system against evaluation criteria
as specified in Table 13. Exceptional proposals with regard to each criteria area shall receive a high
amount of points for that criteria. Unacceptable or substandard proposals with regard to each criteria will
receive fewer or no points for that criteria. NYCERS reserves the right to reduce a Respondent’s total score
by any amount of points, without limitation, as a result of any RFP exceptions included in Respondent
proposals that NYCERS deems, at its sole discretion, to be unfavorable to NYCERS.
Table 13. Evaluation Criteria

Criteria Weighting
Organization Capability and Relevant Experience (quality, track 20%
record, and financials)
Proposed Services (quality, approach, and methodology) 35%
Proposed Project Staffing and Resource Management (quality of 25%
both staffing plan and individual proposed staff members)
Cost 20%
Total: 100%

I.3. Basis of Award

Any contracts awarded will be on the basis of the proposals determined to be the most advantageous to
NYCERS, taking into consideration the price and such other factors or criteria that are set forth in this RFP.

RFP ID # 04282020-LRP Page 46

 Software Quality Assurance and Validation Services for the LRP

NYCERS reserves the right to evaluate and/or reject all bids, in whole or in part, and to waive or modify
technicalities, irregularities, and omissions, or solicit new proposals if, in NYCERS’ judgment, the best
interest of NYCERS will be served. Following bidding selection and prior to signing a contract, NYCERS
reserves the right to further negotiate cost or other specifics. In the event the vendor is unable to fulfill their
agreement with NYCERS, in part or in whole, NYCERS reserves the right, at its sole discretion, to evaluate
a replacement from any other Respondents or seek the services from the Respondent with the next highest
Final Composite Score.
Respondents are hereby notified that the prices stated in their proposals are irrevocable until the contract is
awarded, unless the proposal is withdrawn. Offers may be withdrawn in writing to NYCERS no earlier than
90 days from opening and in advance of the award. NYCERS reserves the right to request a cost
breakdown of the Respondent’s proposed price.

RFP ID # 04282020-LRP Page 47