Evaluation of Innovation Excellence Indicators of Public funded R&D Organisations – Round 2.0

Download Manual

A Manual for Participating Organisations

Prepared on the behalf of the Office of the Principal Scientific Adviser

Introduction

At the behest of the Prime Minister’s office, the Office of the Principal Scientific Adviser to the Govt of India (o/PSA) in July 2019, initiated a first of its kind exercise for capturing and evaluating the innovation indicators of public funded R&D organisations. On completion of the exercise in March 2022, a two-part report titled “Evaluation of Innovation Excellence Indicators of Public Funded R&D organisations” was prepared.

The reports were submitted to the Prime Minister’s office (PMO) in Feb 2022 and were duly considered. PMO directed o/PSA to convene a meeting with stakeholder ministries / departments to (i) sensitise them to the findings of the study and (ii) request more active participation in Round 2 of the study.

With this background, and consequent to the direction of the PMO and envisaging wider participation of public funded R&D organisations, round 2 of the study has been initiated. The purpose of this study is to capture in much more granularity the innovation indicators and the research being undertaken by various public funded R&D organisations to help gauge the performance of these labs with respect to their socio-economic contribution, STI excellence, and organisational capabilities and practices as well as to cover a wider number of public funded R&D labs.

This framework has been further developed into appropriate survey instruments for online data collection through a web portal (http://www.indiascienceindicators.gov.in/). The purpose of this manual is to provide information and context of this framework and guidance on survey instruments to members of participating organisations.

The Office of the Principal Scientific Adviser is the main implementing body for this research, with the Confederation of Indian Industries (CII) as designated knowledge partner. The Centre for Technology, Innovation and Economic Research (CTIER) is providing knowledge support.

About the Framework

Although various R&D labs work on diverse areas, they can be grouped into three categories i.e. Basic Research Labs, Applied Research Labs, and Service Labs. It is hoped that for the organisations themselves, the framework provides an opportunity to benchmark their performance against other organisations in their respective category and identify interventions that may be required to improve their performance. The framework also provides an opportunity for these labs to identify areas of untapped potential and discover interventions to improve their performance in their respective areas.

The framework (see Table 1 below) has three main pillars common to all three categories, wherein each sub-pillar has a number of indicators and a matrix of scoring parameters with different weights to capture relevance under the three categories of R&D Labs.

Table 1: Overview of the Framework
Outcomes Outputs Inputs
Pillars Socio Economic Impact Science, Technology and Innovation Excellence Organisational Effectiveness
Sub-Pillars
  1. Contribution to India’s SDGs and national programmes

  2. Employment generation and human resources development

  1. Scholarly research output and quality

  2. Development and innovation output and quality

  3. Commercialisation of technologies and revenue generation

  4. Collaborative research

  1. Resource management

  2. Governance

  3. Equity, diversity and inclusion

  4. Internal capacity building

Based on this framework, three separate questionnaires have been developed for online deployment, one for each category. Each questionnaire differs slightly based on category relevant indicators.

Nomination of Data Officers

All participating organisations are required to nominate a Data Officer to coordinate this exercise 5 as the designated central point of contact for all future correspondence/ engagements.

The Data Officer must have a deep, overall understanding of the organisation’s work. The Data Officer must be well placed to coordinate with different departments within the organisation to collect data internally as per the requirements of the framework. For example, the Data Officer will have to coordinate with the finance department to gather data related to funding and earnings or with the administrative department to gather data related to the total scientific staff present at the organisation. Similarly, for gathering IPR related data, the Data Officer may have to coordinate with more than one department within the organisation. Hence, the selection of a Data Officer who has a bird’s eye view of the organisation’s work and can coordinate within the organisation is critical.

Importantly, the Data Officer will also be responsible for presenting and getting data duly vetted by the Director of the organisation before final submission. It is recommended that the Data Officer attend the orientation webinar held by knowledge partners before embarking on this exercise.

Registration on the Web Portal

Registration on the web portal (http://www.indiascienceindicators.gov.in/) is mandatory. The The Data Officer may use his/her email address for the purpose of registration.

Please follow the following steps for registration:

  1. Please click on ‘Registration’ to register your institution on the portal

  2. Please enter all the information along with the lab type or category, defined as follows:

Category Definition
Basic R&D Basic research is experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundation of phenomena and observable facts, without any particular application or use in view.
Applied R&D Applied research encompasses original investigation 6 undertaken in order to acquire new knowledge. lt is, however, directed primarily towards a specific, practical aim or objective.
Services R&D Service R&D is systematic work, drawing on knowledge gained from research and practical experience and producing additional knowledge, which is directed to producing new products or processes or to improving existing products or processes.
  1. You may choose to respond to more than one category, if it applies to your organisation

  2. Please note you are required to fill in questionnaires and upload supporting documents for each lab type that you have chosen - for example, if your organisation’s work covers the attributes of both basic R&D and applied R&D, then you will fill the questionnaires for basic R&D and applied R&D

  3. Please verify your account from the verification link sent to you on email

  4. After verification, you will receive login details on the email ID registered on the portal

  5. Once you login, please click on the ‘Questionnaire’ tab, where you will be able to see the questionnaires for the ‘category’ selected by you in Step 2

  6. You can save and log out of your account at any time. Your responses will be saved and you will be able to log back into the survey and complete more responses at a later time.

About the Questionnaire

  1. General

    Please bear in mind the following points when filling the questionnaire:

    1. Please note that all questions and the section on organisation details of the lab are mandatory
    2. All data you enter will be kept confidential
    3. You are required to fill in data for the financial years 2021-22 and 2022-23 except for publication related questions where data is entered for the calendar years 2021 and 2022
    4. You will require access to Clarivate Analytics Web of Sciences and Incites or Scopus and 7 Scimago to report publications related data
    5. You are required to upload supporting documents for some questions for validation of the aggregate data reported on the web portal - refer to next section for more details on supporting documents
    6. The actual time taken for completing the questionnaire on the portal is between 45 minutes to an hour. However, time taken to collect data within the organisation may vary depending on existing systems within the organisation
    7. It is recommended that the nodal officer first go through the PDF versions of the questionnaires, identify questions where support will be required from other departments within the organisation and then proceed to the online questionnaires.
    8. Refer to the definitions and explanations given under questions before formulating responses
    9. You will be able to check all your responses at one go before final submission/ You will be able to save a copy of the responses for your records at the time of submission.
  2. Supporting Documents

    Please note that it is mandatory to upload all supporting documents for the system to accept your final submission.

    1. Please upload all supporting documents for each category if responding to more than one category
    2. Supporting documents are templates in downloadable format and will provide the responses to be filled in the questionnaire for the relevant questions
    3. Microsoft Excel 2021 is required for filling in the templates
    4. Note that one template may cover multiple questions - refer to section III ‘Templates for Supporting Documents’ for questions that are covered by a corresponding template
    5. You will see a pop-up reminder to upload the supporting documents. You can also come back later to the question to upload the supporting documents
    6. You will see a checkbox below the questions that are covered by templates to confirm the response matches the data entered in the template
    7. Please note that there are a total of 19 templates in downloadable format, of which 16 are the same for all 3 categories - please refer to section III ’Templates for Supporting Documents’ for these 16 common templates
    8. You will see a pop-up reminder to upload the supporting documents. You can also come back later to the question to upload the supporting documents
    Important Note:
    • All data entered on the supporting documents is used for data validation, hence it is advisable that you enter data on the templates before entering corresponding responses to reduce manual error
    • All templates are coded to provide you with the required response once you enter the data
    • Kindly do not tamper with the format of the template.
  3. Templates for Supporting Documents
    Template Name Basic labs (Q. No.) Applied labs (Q. No.) Services labs (Q.No.)
    1 - Technologies and SDGs Q1 Q1 Q1
    2 - Projects Executed Q2 Q2 Q2
    3 - Workforce Q7 Q7 Q8
    4 - Startups Support and Exit Q8 Q8 Q9
    5 - Spinouts Q13 Q13 Q14
    6 - Human Resources Generated Q14 Q14 -
    7 - Awards and Fellowships Q16 Q16 Q18
    8 - Publications 18a 18a 20a
    9 - Commissioned Reports 18b 18b 20b
    10 - Recognitions - - Q21
    11 - Report Leading to Designs and Products - - Q22
    12 - IPR Filed Q21a Q21a Q23
    13 - IPR Granted Q22a Q22a Q24
    14 - Technology Transfer Q24a Q24a Q26a
    15 - Non-worked patents Q24b Q24b Q26b
    16 - New Services and Products Q27 Q27 Q29
    17 - Open Research and Testing Facilities Q52 Q52 Q54
    18 - Career Development Programmes Q61 Q61 Q63
    19 - Support to Researchers Q62 Q62 Q64
  4. Type of Questions

    The questionnaires for basic and applied labs have a total of 62 questions while the questionnaire for services labs has 64 questions. Some questions may have sub-questions. Some questions allow for choosing more than one option. Relevant explanatory notes, instructions and FAQs have been provided for each question.

    There are three types of questions:

    1. Numeric: These questions require either a response in percentage terms or absolute numbers.
    2. Binary: These questions require a Yes/No response
    3. Qualitative: A few questions are subjective in nature and require description. These questions will be assessed by a Committee of domain experts.
  5. Key Definitions and Explanations

    A preliminary list of key terms and respective definitions and explanations is included below. This list will be updated periodically on the web portal to aid users based on frequently asked questions.

    Terms

    Definitions

    Basic labs

    Basic research is experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundation of phenomena and observable facts, without any particular application or use in view.

    Applied labs

    Applied research is original investigation undertaken in order to acquire new knowledge. lt is, however, directed primarily towards a specific, practical aim or objective.

    Services labs

    Service R&D is systematic work, drawing on knowledge gained from research and practical experience and producing additional knowledge, which is directed to producing new products or processes or to improving existing products or processes.

    Projects Executed

    Projects executed in a particular year would include projects started in the relevant year or completed in the relevant year. They would also include multi-year projects that may have started in a previous financial year and are on-going in the relevant year. Please include all projects that have been undertaken either as a standalone project or those falling under particular themes or programmes.

    Researchers/ Scientific Staff

    Researchers include the following positions:
    - Scientific staff (Scientist B or equivalent and above) in

Guidelines

Feedback

As task of this nature is being undertaken on a national scale, there are bound to be challenges and useful feedback from the participating organisations.
You may use the comment box provided on the web portal to convey any suggestions or pertinent comments regarding your responses.
Additionally, if you wish to raise a query regarding specific questions or submit your feedback, please contact Dr. Rahul Katna(rahul.katna@cii.in).

Loading...