Evaluation of Science Indicators of Public Funded R&D Institutions


A Manual for Participating Organisations


Prepared on the behalf of the Office of the Principal Scientific Adviser



Introduction

The Office of the Principal Scientific Adviser (PSA) at the instance of the PMO has been entrusted
to design a functional framework for ascertaining the absolute and relative strength and
weaknesses of our public funded R&D organizations and in particular:

1. To determine whether the institutions are working as per their mandate or the mandate itself
needs a revision in line with present priorities and technologies

2. To assess required interventions for improvement

3. To ascertain whether institutions are delivering outcome-oriented R & D

4. To assess productivity of research institutions in terms of optimal utilisation of resources

5. To derive actionable policy recommendations from the learning from the evaluation
process.

This framework has been further developed into appropriate survey instruments for online data
collection through a web portal (http://www.indiascienceindicators.gov.in/).

The purpose of this manual is to provide information and context of this framework and
guidance on survey instruments to members of participating organisations.

The Office of the Principal Scientific Adviser is the main implementing body for this research,
with the Confederation of Indian Industry (CII) as designated knowledge partner. The Centre for
Technology, Innovation and Economic Research (CTIER) is providing knowledge support.

About the Framework

A Task Force under Prof. Goverdhan Mehta was constituted by NITI Aayog to develop this
framework in consultation with Directors of R&D labs across various ministries and departments.
It was noted that although various R&D labs work on diverse areas, but they can be grouped into
three categories i.e. Basic Research Labs, Applied Research Labs and Service Labs. It is hoped
that for the organisations themselves, the framework provides an opportunity to benchmark their
performance against other organisations in their respective category and identify interventions that
may be required to improve their performance.

The framework (see Table 1 below) has three main pillars common to all three categories, wherein
each sub-pillar has a number of indicators and a matrix of scoring parameters with different
weights to capture relevance under the three categories of R&D Labs


Table 1: Overview of the Framework
Outcomes Outputs Inputs
Pillars Socio Economic
Impact
Science, Technology and
Innovation Excellence
Organizational
Effectiveness
Sub-
Pillars
1. Contribution to India
SDGs and National
Programmes

2. Employment
Generation and
Human Resources
Development

1. Scholarly Research Output
and Quality
2. Development and
innovation Output and
Quality
3. Commercialisation of
technologies and revenue
generation
4. Collaborative Research
1. Mandate Alignment
2. Resource
Management
3. Governance
4. Equity, Diversity
and Inclusion
5. Internal Capacity

Based on this framework, three separate questionnaires have been developed for online
deployment, one for each category. Each questionnaire differs slightly based on category relevant
indicators.

Nomination of Nodal Officer

All participating organisations are required to nominate a nodal officer to coordinate this exercise
as the designated central point of contact for all future correspondence/ engagements.
The nodal officer must have a deep, overall understanding of the organisation’s work. The nodal
officer must be well placed to coordinate with different departments within the organisation to
collect data internally as per the requirements of the framework. For example, the nodal officer
will have to coordinate with the finance department to gather data related to funding and earnings
or with the administrative department to gather data related to the total scientific staff present at
the organisation. Similarly, for gathering IPR related data, the nodal officer may have to coordinate
with more than one department within the organisation. Hence, the selection of a nodal officer who
has a bird’s eye view of the organisation’s work and can coordinate within the organisation is
critical.
Importantly, the nodal officer will also be responsible for presenting and getting data duly vetted
by the Director of the organisation before final submission.
It is recommended that the nodal officer attend the orientation webinar held by knowledge partners
before embarking on this exercise.

Registration on the Web Portal

Registration on the web portal (http://www.indiascienceindicators.gov.in/) is mandatory. The
nodal officer may use his/her email address for the purpose of registration.
Please follow the following steps for registration:
1. Please click on ‘Registration’ to register your institution on the portal
2. Please enter all the information along with the lab type or category, defined as follows:

Category Definition
Basic R&D Basic research is experimental or theoretical work undertaken
primarily to acquire new knowledge of the underlying foundation of
phenomena and observable facts, without any particular application
or use in view
Applied R&D Applied research is original investigation undertaken in order to
acquire new knowledge. lt is, however, directed primarily towards a
specific, practical aim or objective.
Services R&D Service R&D is systematic work, drawing on knowledge gained from
research and practical experience and producing additional
knowledge, which is directed to producing new products or processes
or to improving existing products or processes.

3. You may choose to respond to more than one category, if it applies to your organisation
4. Please note you are required to fill in questionnaires and upload supporting documents for
each lab type that you have chosen - for example, if your organisation’s work covers the
attributes of both basic R&D and applied R&D, then you will fill the questionnaires for
basic R&D and applied R&D
5. Please verify your account from the verification link sent to you on email
6. After verification, you will receive login details on the email ID registered on the portal
7. Once you login, please click on the ‘Questionnaire’ tab, where you will be able to see the
questionnaires for the ‘category’ selected by you in Step 2
8. You can save and log out of your account at any time. Your responses will be saved and
you will be able to log back into the survey and complete more responses at a later time.

About the Questionnaire

I. General

Please bear in mind the following points when filling the questionnaire:

1. Please note that all questions and the section on organization details of the lab are
mandatory
2. All data you enter will be kept confidential
3. You are required to fill in data for the financial years 2017-18, 2018-19 and 2019-20 except
for publication related questions where data is entered by calendar years 2017, 2018 and
2019
4. You will require access to Clarivate Analytics Web of Science and Incites or Scopus and
Scimago to report publications related data
5. You are required to upload supporting documents for some questions for validation of the
aggregate data reported on the web portal - refer to next section for more details on
supporting documents
6. The actual time taken for completing the questionnaire on the portal is between 45 minutes
to an hour. However, time taken to collect data within the organisation may vary depending
on existing systems within the organisation.
7. It is recommended that the nodal officer first go through the PDF versions of the
questionnaires, identify questions where support will be required from other departments
within the organisation and then proceed to the online questionnaires.
8. Refer to the definitions and explanations given under questions before formulating
responses
9. You will be able to check all your responses at one go before final submission/ You will
be able to save a copy of the responses for your records at the time of submission

II. Supporting Documents

Please note that it is mandatory to upload all supporting documents for the system to accept
your final submission.
1. Please upload all supporting documents for each category if responding to more than one
category
2. Supporting documents are of three types: templates in downloadable format (in excel),
screenshots and policy document
3. Note that one template may cover multiple questions - refer to the guide ‘Templates for
Supporting Documents’ in the next section for questions that are covered by a
corresponding template
4. There are a total of 18 templates in downloadable format, of which 13 are the same for all
3 categories - please refer to the guide for these 13 common templates
5. You will see a pop-up reminder to upload the supporting documents. You can also come
back later to the question to upload the supporting documents
6. Please ensure that all supporting documents are uploaded correctly where requested

Important Note:
· All data entered on the supporting documents is used for data validation, hence it is
advisable that you enter data on the templates before entering corresponding responses
to reduce manual error
· You must not change the format of the template while entering data


III. Templates for Supporting Documents

Template Name Basic labs (Q. No.) Applied labs (Q. No.) Services labs (Q.No.)
1 - Technologies and SDGs Q1 Q1 Q1
2 - Projects Executed Q2, Q31, Q32, Q34 & Q35 Q2, Q31, Q32, Q34 & Q35 Q2, Q32, Q33, Q35 & Q36
3 - Workforce Q8, Q37, Q41, Q56 & Q57 Q8, Q37, Q41, Q56 & Q57 Q9, Q41, Q56 & Q57
4 - Startups incubation & Exit Q9 & Q10 Q9 & Q10 Q10 & Q11
5 - Employment generated by startups Q11 Q11 Q12
6 - Consultancies for startups Q12
7 - Human resources generated Q13 Q12 u
8 - Awards and Fellowships Q16, Q17 Q15, Q16 Q16, Q17
9 - Publications Q18a, Q33 & Q36 Q17, Q33 & Q36 Q18a, Q34 & Q37
10 - Commissioned Technical Reports Q18b Q18 Q18b
11 - Technology documents prepared Q19
12 - Recognitions Q20
13 - Reports leading to designs and products Q21
14 - IPR Filed Q21 Q21 Q22
15 - IPR Granted Q22 Q22 Q23
16 - Technologies Transferred Q23 & Q25 Q23 & Q25 Q24 & Q26
17 - New services and products Q26 Q26 Q27
18 - Outside researchers Q52 Q52 Q52

IV. Type of Questions

Each questionnaire has a total of 62 questions. Some questions may have sub-questions.
Some questions allow for choosing more than one option. Relevant explanatory notes,
instructions and FAQs have been provided for each question.
There are three types of questions:
1. Numeric: These questions require either a response in percentage terms or absolute
numbers.
2. Binary: These questions require a Yes/No response
3. Qualitative: A few questions are subjective in nature and require description. These
questions will be assessed by a Committee of domain experts.

V. Key Definitions and Explanations

A preliminary list of key terms and respective definitions and explanations is included
below. This list will be updated periodically on the web portal to aid users based on
frequently asked questions.

Terms Definitions
TRL 1 Basic principles observed and reported.
TRL 2 Technology concept and/or application formulated.
TRL 3 Analytical and experimental critical function and/or characteristic proof-of-concept.
TRL 4 Technology basic validation in a laboratory environment.
TRL 5 Technology basic validation in a relevant environment.
TRL 6 Technology model or prototype demonstration in a relevant environment.
TRL 7 Technology prototype demonstration in an operational environment.
TRL 8 Actual technology completed and qualified through test and demonstration.
TRL 9 Actual technology qualified through successful mission operations
R&D and S&T Budget These include R&D and scientific budget related costs whereas running costs or recurring costs (for example electricity, rent etc.) that would constitute administrative costs would be excluded. If any of the costs mentioned are directly project related (for example travel for research, conferences, seminars and workshops), they would be included as R&D budget.
Total Staff Total Staff at Laboratory includes permanent scientists, contractual researchers, technical support staff, administrative staff
Scientific and Research Staff Scientists and researchers include permanent scientists (Scientist B/Level 10 or equivalent and above) and contractual researchers (researchers hired for projects, JRFs, SRFs and other fellowship awardees, etc.).
Projects Executed Projects executed in a particular year would include projects started in the relevant year or completed in the relevant year. They would also include multi-year projects that may have started in a previous financial year and are on-going in the relevant year. Please include all projects that have been undertaken either as a standalone project or those falling under particular themes or programmes.
Start-ups incubated in the premises of your organisation An incubated startup will have access to all incubator facilities like land, equipment, research support, mentoring, auxiliary/technical support such as marketing, accounting, legal help etc.
Incubated startups successfully exited Successful exits are those that have graduated from the incubation program of the organisation under organisation's policy, except those that are compulsorily retired/ removed/ terminated under the organisation's Policy.
Commissioned technology development/ design/project reports prepared by your organisation These reports include detailed process technology report, process design report, process equipment design & development, toxicological studies, detailed process control & instrumentation scheme for a technology, design of an automated production etc. commissioned by the Government of India, State Governments, Public Sector Undertakings and private sector firms.
Number of citations received by papers published Please consider papers published in the previous three years, (eg, when reporting data for the year 2019 use citation counts received in 2019 of papers published in 2016, 2017 & 2018).
Percentage of publications in top 10% of journals as per Impact Factor by Subject Category Please consider the top 10% Journals as per Impact Factor by subject category - for each subject that your organisation publishes in. Report the percentage of your publications in these top 10% journals as a share of your total publications.
Technologies transferred These include technologies that may be transferred through direct sale, license, spinoffs or transfer for use at cost or free of cost.
Technology documents prepared Reports include detailed process technology report, process design report, process equipment design & development, toxicological studies, detailed process control & instrumentation scheme for a technology, design of an automated production etc. commissioned by the Government of India, State Governments, Public Sector Undertakings and private sector firms.


Feedback

As this is the first time a task of this nature is being undertaken on a national scale, there
are bound to be challenges and useful feedback from the participating organisations. You
may use the comment box provided on the web portal to convey any suggestions or
pertinent comments regarding your responses.
Additionally, if you wish to raise a query regarding specific questions or submit your
feedback, please contact Ms. Divya Arya (divya.arya@cii.in) or Ms. Namita Bahl
(namita.bahl@cii.in).

Loading...