Prepare and submit your replication package

Once your handling co-editor has conditionally accepted your paper (prior to their review of your final response and final acceptance), your paper will undergo our reproducibility checks. As a result, your last resubmission will be done at the Data Editor's Editorial Express portal, and you will need to submit your replication package alongside your paper and approved online appendices. To ensure the proper handling of your submission, it is crucial that you carefully follow the instructions provided below. This page offers comprehensive information to assist you in preparing your replication package. We strongly recommend that you thoroughly review this information before submitting your materials to ensure a smooth and fast process.

1. Preliminaries
2. Contents of the package
3. README file
4. Data Citations
5. Best practices
6. Submit your replication package
7. Publish your accepted replication pacakge

Preliminaries

Submit your replication package on a zip folder that includes all the necessary files and directories that allow reproducing all the results in the paper and approved online appendices. Do not include the paper or the online appendices inside this zip file! Make sure that you include all the required items outlined our Data and Code Availability Policy.

In cases where a granted exemption allows you to exclude some files from the public replication package, sharing these files confidentially with us, please supply these files in a separate and clearly labeled zip file (e.g. "Confidential data not for publication").

During the submission process, you will be prompted to include a fully filled and signed checklist, which is useful for you to review that you include all the required items. Please ensure that you download, complete, and attach the checklist accordingly.

For your convenience, find here the key external links included in this page (you still need to carefully read the rest of the page!):

Submit your package Checklist  Template of README Guidance on data citations Data and Code Availability Policy ES Journals' Community at Zenodo Terms of use of commonly used datasets

Contents of the package

As indicated in our Data and Code Availability Policy all packages should include the following components:

• README File: Please include a README file in PDF format (README.pdf). To ensure that all necessary information is covered, we highly recommend using the Social Sciences Data Editor's template of README, which includes all the required items outlined in our policy. You can also create your README file conveniently using this web app.
• Raw datasets: Include the raw datasets used in your paper and approved online appendices. It is important that these datasets are accompanied by comprehensive, transparent, and precise documentation that describes all variables used. In cases where a data exemption was granted (more here), you must either provide the replication team with (or help them obtain) temporary access to the restricted data for reproducibility checks, or else include a synthetic/simulated dataset in your package. This synthetic/simulated dataset should allow running the code and producing all outputs as presented in the paper and appendices, but does not need to be designed to reproduce the exact results in the paper.
• Analysis data? Include your intermediately created analysis data only if the provided data and codes are unable to fully recreate them within a reasonable time frame.
• Data cleaning codes and analysis codes: Include the data cleaning codes and analysis codes necessary to generate all reproducible outputs reported in the paper and approved online appendices. A few important notes on these codes:
  • They should be provided in a source format that can be directly interpreted or compiled by the appropriate software.
  • It is highly encouraged to include master files that run all the code from raw data to final results, especially if there are numerous scripts or complex/time-consuming human steps involved.
  • The codes should save all exhibits (tables, figures, numerical information in text, etc.) from the paper and authorized appendices in a specified directory within the replication package.
  • If your codes are written in compiled languages, please provide precise instructions for all steps and compiling options in the documentation. Including a make file that reproduces the compilation steps is strongly encouraged.
  • We discourage the use of software that does not allow output generation using scripts (e.g., ArcGIS). If such software is used, detailed and explicit step-by-step instructions must be included in the README file. These instructions should enable non-highly trained users to exactly reproduce the generated outputs independently of the authors.
• Data in non-proprietary format:  If your data is provided in a proprietary format (e.g., Stata's .dta), please include a copy of the data in a non-proprietary format (e.g., ASCII, CSV).
• Additional documentation for primary collected surveys or experimental data: If applicable, include any additional documentation required for primary collected surveys or experimental data as specified in Rule 10 of our Data and Code Availability Policy.

README file

The README file is crucial for ensuring that all users, at the level of an advanced PhD student and above, can successfully reproduce the results in the paper within a reasonable amount of time independently of the authors. To assist you in creating the README file, we strongly recommend using the Social Sciences Data Editor's template of README, which includes all the required items outlined in our policy. Additionally, you can utilize this web app as a convenient tool for creating your README file. The minimum required information for the ReadMe file includes:

• Description of package content: Clearly outline the contents of the package, including datasets, programs, folders, and any other relevant components. Connect each provided data file with the corresponding data source.
• Data Availability Statement (DAS): Provide precise instructions on how the data were obtained, including any required registrations, memberships, application procedures, monetary and time costs, qualifications, and if applicable, include the URL to download the data (often part of the data citation). The DAS should include sufficient detail for independent researchers to replicate the necessary steps to access the original data. When applicable, the DAS should also specify the specific version of the dataset and the original date of access by the authors. Similarly, the DAS should clearly indicate which datasets are included and excluded from the replication package.
• Code execution instructions: Include precise human-readable instructions on how to run the code, ensuring that users can easily follow the steps to reproduce the results. When you use software that does not allow output generation using scripts (e.g., ArcGIS) include detailed and explicit step-by-step instructions that enable non-highly trained users to exactly reproduce the generated outputs independently of the authors.
• Computational requirements, including libraries and packages and running time: Specify the software and hardware used in the package, including software versions, minimum hardware requirements to run the codes, and expected running times. When hardware, software requirements, and/or running times are heterogeneous across diferent portions of the package, specify the requirements for each of the different parts must be indicated. List all packages and libraries that need to be installed in order to run the code, including version. Provide clear instructions on how users can obtain and install these libraries.
• List of tables and figures: Provide detailed indications on where the outputs produced by the code are saved and on how to map each of these outputs to the exhibits included in the paper and approved online appendices.
• Data Citations: Include proper data citations for all datasets used in the paper and approved online appendices. Just like citing other papers, include these citations in a dedicated references section at the end of the README file following the citation formats specified by each journal, ensuring that references can be accurately indexed by bibliometric search engines.

Data citations

All data used in the paper and the approved online appendices must be appropriately cited in both the paper/appendices and in a dedicated references section of the README file. As a general guideline, citations of data employed in the paper should be included in the paper's references section, while citations exclusively pertaining to data used in the approved online appendices may be relegated to the appendix. However, in exceptional circumstances, such as when there is a large number of data sources to cite or when recommended by the handling co-editor, citations of data used in the paper may be included in a references section of the approved appendix. The citations included in the references section of the README file should follow the citation formats specified by each journal, ensuring that references can be accurately indexed by bibliometric search engines.

When utilizing data that is part of another paper's replication package, it is important to cite both the original paper and the replication package. It is equally essential to cite commonly used datasets, as the funding support for public and private institutions, even for widely employed datasets, relies significantly on data citations.

For citing your own data, adhere to the same citation style used for other datasets. In the paper, include the citation, but there is no need to self-cite your own data in the README file. It is worth noting that your replication package will acquire a DOI only after it is posted in the Econometric Society Journals' Community at Zenodo. To ensure a DOI for your package before submission, it is advisable to initiate a deposit at Zenodo beforehand to reserve a DOI. This way, the deposit, along with the secured DOI, can be completed after the reproducibility checks are concluded.

Check for additional guidance on data citations at the Social Sciences Data Editors site.

Best practices

In order to enhance the quality and usability of replication packages, it is highly recommended to follow a set of best practices. Although you might encounter some of these practices late in the production process, we strongly encourage you to take note for your future research endeavors. Here are some key recommendations:

• Preserve the integrity of your raw data by keeping it in a read-only format. Generate separate analysis datasets for conducting your analyses.
• Utilize programs to clean your raw data and generate your analysis data, as many journals require their submission along with the paper.
• Maintain a record of the date and versions of the datasets you obtained, as datasets can undergo changes over time and it may become challenging to retrieve specific versions in the future.
• Organize your directory structure by separating code and data. This will facilitate version control of your code and simplify the creation of a replication package.
• Prefer using programs or scripts instead of relying solely on command-line instructions.
• Design your code to be executed as a cohesive unit rather than section by section. Create a master file that calls all subsidiary files. When using compiled languages, use make files that can reproduce the compilation steps and compiler options.
• Utilize log-files to store the output generated by your code, ensuring that it is saved and not solely displayed during runtime.
• Modify your code to produce tables and figures that match those presented in the manuscript. This approach will save time during the replication process.
• Set paths at the beginning of your master file and use relative paths or global variables, reducing the need for users to modify path references throughout your code.
• Whenever possible, use a path format that is compatible across different operating systems. For instance, in Stata, use forward slashes ("/") to separate directories, even if you are using Windows.
• Prior to submission, run your code from the replication folder, ideally on different machines, to verify that it executes successfully and reproduces all results.
• Choose meaningful names for your files, particularly when generating tables and figures as separate files. Additionally, consider naming your master file as "Main", "Master" or something similar to make it easy to identify.
• Create sub-folders within your package when dealing with a large number of files. Ensure that your package includes all relevant folders, even if some of them are initially empty and will be populated with program outputs.

The Data Editors of the Econometric Society (Joan Llull), the American Economic Association (Lars Vilhuber), the Review of Economic Studies (Miklós Koren), the Economic Journal and Econometrics Journal (Florian Oswald), and the Canadian Journal of Economics (Marie Connolly) are coordinating reproduciblity requirements and guidelines across different journals. An example of this cooperation is this common guidance by Social Science Data Editors.

Submit your replication package

Submit your replication package via Editorial Express at the following link:

Submit your package

• Identifying information on your submission (Title, Journal, MS number, RP number): This information is crucial to correctly trace your paper in the system. You have to keep track of two important numbers: the manuscript (MS) number from your original submission to one of the Econometric Society journals, and the replication package (RP) number, which will be assigned to you after you complete the submission of your package for the first time. You will always be required to input the MS number, and in re-submissions of the package, also the RP number.
• Manuscript and approved online appendices: Include them as a PDF. They will be transferred to your Handling Editor directly when the reproducibility checks have been completed.
• Checklist: Download, fill, and sign this checklist (required), which will help you check that you included all the required elements of the replication package.
• Cover letter to the Handling Editor: Include the cover letter addressed to the Handling Editor in response to the conditional acceptance decision. This letter will be transferred at the end of the checks. Include it in all re-submissions of your replication package. If your paper or approved online appendices are modified (even if marginally) as a result of the reproducibility checks, clearly state all changes in this cover letter as well as in the cover letter to the Data Editor. Note that the Handling Editor will only read your cover letter after the reproducibility checks are concluded.
• Cover letter to the Data Editor: In the first submission of your replication package, provide any information necessary regarding how to re-assemble the replication package if you needed to submit it in more than one file, how to access the confidential data if applicable, and any other consideration that yo u may consider relevant. In the subsequent re-submissions, clearly indicate how you addressed the comments included in the replicators' report and the decision letter of the Data Editor. If your paper or approved online appendices are modified (even if marginally) as a result of the reproducibility checks, clearly state all changes in this cover letter as well as in the cover letter to the Handling Editor.
• Replication package: Ideally, send the (public) replication package in a single zip file. If you share with us your confidential data directly, include it on a separate zip file. Editorial Express does not allow files over 100MB. If your package is larger, try splitting it in multiple files (you can include up to three). If it is still too large, please contact the Data Editor Office at dataeditor@econometricsociety.org. Note that file names are modified by Editorial Express, so please use the description line to indicate the original file name and any other relevant information (e.g. to identify confidential data or to include instructions to re-merge the submitted files).

Publish your accepted replication package

Once the reproducibility checks are concluded, you will be requested to do an additional step: to publish your replication package at the Econometric Society Journals' Community at Zenodo. To do so, you need to complete the following steps:

1. If you don't have a Zendo account to publish your files, you may sign up here following very simple steps.
2. Go to the Econometric Society Journals' Community at Zenodo at https://zenodo.org/communities/es-replication-repository.
3. Start a new upload within the community using the top-right green button in the screen.
   
4. Compress your replication package (the exact folder "Replication package" that went through the reproducibility checks) on a zip file. Drag-and-drop the zip file to upload it. Make sure you only publish the replication package and not the paper or online appendices. Especially, do not publish the content of the "Confidential data not for publication" folder!
   
5. Click on the Start upload green button to upload.
   
6. If you didn't skip step 2, your upload is already linked to the Econometric Society Journal's Community.
   
   If your upload is not directly linked to the Econometric Society Journals' community, you can search the community on the search bar.
   
7. Fill the required metadata:
   • Upload type: Software (or Dataset if you more applicable).
   • Publication date: leave the default today's date.
   • Title: Replication package for: "{full title of your manuscript}".
   • Authors: Typically the same authors as the manuscript, but you can add others who have contributed significantly to the code you are uploading, such as research assistants (in that case, you are making them coauthors of your replication package, but not of the paper).
   • Description: write a full citation to your manuscript as a description of what the replication package contains.
   • Other optional metadata: don't worry about this, you can leave it blank.
   
8. Select an open-access license for your upload.
   
9. Save the "Digital Object Identifier" (DOI) issued by Zenodo (see the image above) and email it to the Editorial Office at dataeditor@econometricsociety.org. This DOI will appear in your published paper, and will hence link your article and your package.
10. Save and then Publish using the corresponding buttons that appear both at the bottom and at the page. This will notify the Editorial Office who will review that the files you submitted correspond to those that went to the reproducibility checks.
    

IMPORTANT NOTE: once you press publish, you will not be able to edit or delete your uploaded files. You can create new versions of your upload at any point in the future, but these versions will have a different DOI, so they will be linked to your earlier and future versions of the package, but they will not be directly linked to your paper. Please double-check that you publish the files that went through the reproducibility checks, that you are not publishing the paper or online appendices, and that you are not publishing the content you submitted in the folder "Confidential data not for publication".