Organizing a workshop

by Michael Ernst

June, 2006
Edited by Dan Grossman, August 2007

Contents:

This document is intended to aid the PASTE co-chairs in organizing the workshop with a minimum of hassle and mistakes. It might also be helpful to organizers of other workshops, especially ACM-sponsored ones. Much of the document is a list of tasks that the co-chairs must perform, listed in roughly chronological order, with some tips about how to avoid mistakes while doing them.

Please improve this document it by offering additions, corrections, and other suggestions — whether you are a PASTE organizer or not.

This document is accompanied by a collection of materials from previous PASTE workshops: proposals, budgets, lists of addresses to use for publicity, workshop reports, etc. Those materials are available only to the PASTE co-chairs, and by request; they are not publicly available.

Also consider reading my advice on running a PC meeting.

PASTE organization and character

PASTE is run by two co-chairs who share the work, functioning jointly as both program and general chairs. (There is no separate general chair of PASTE.) Both co-chairs should be eminent researchers in the intersection of programming languages and software engineering. By tradition, one chair leans more toward the programming languages community, and one leans more toward the software engineering community.

PASTE is co-located with PLDI and FSE in alternate years, starting with PLDI 1998. Most of the logistical issues are taken care of by the main conference and ACM, so most of the work required of the co-chairs is program-related. It works well to split tasks (publicity, financial, proceedings, etc.) between the chairs — but do check up on each other once in a while!

The PASTE steering committee consists of the chairs of the previous two instances of PASTE, plus the vice chairs of the sponsoring organizations, ACM SIGPLAN and ACM SIGSOFT. The chair is one of the previous year's organizers, typically rotating between the “SE” person and the “PL” person. The steering committee provides advice and counsel, but is not actively involved in operational matters, except that it selects the co-chairs for the workshop.

The steering committee for PASTE 2008 is:

Dan Grossman <djg@cs.washington.edu> (chair; PASTE 2007 co-chair)
Manuvir Das <manuvir@microsoft.com> (PASTE 2007 co-chair)
Michael Ernst <mernst@cs.washington.edu> (PASTE 2005 co-chair)
Thomas Jensen <jensen@irisa.fr> (PASTE 2005 co-chair)
David Rosenblum <d.rosenblum@cs.ucl.ac.uk> (SIGSOFT vice chair)
Chandra Krints <ckrintz@cs.ucsb.edu> (SIGPLAN vice chair)

The steering committee believes it is important for PASTE to be a true workshop. There are a wide variety of possible ways to organize an energetic PASTE workshop program including mixtures of:

ACM guidelines preclude workshops that are exclusively by invitation. However, actively encouraging a number of researchers in software engineering and programming languages to attend PASTE would be helpful in establishing a core group of active participants. Of course, the co-chairs have considerable latitude in determining the form and content of each PASTE meeting, but we hope that these ideas help you understand the diversity of content you might include in a PASTE program.

Proposing and organizing

  1. Read the final reports that were submitted by previous chairs, and skim the other materials that accompany this document.
  2. Read the ACM Conference Planning Guide.
  3. Arrange SIGPLAN and SIGSOFT sponsorship — do this as soon as possible. (Warning: for PASTE 2005, SIGSOFT quickly approved our proposal, but then informed us via physical rather than electronic mail, and the physical mail was misplaced.)
    Initiate the process of obtaining an “ACM X-XXXXX” number, that should be put on the camera-ready versions of the paper, early. It can take a while to obtain the number.
  4. Get agreement from the conference. This is not automatic! More seriously, the organizers may not have institutional memory about the PASTE workshop, which is co-located with conferences every 3 years (or every 6 years, if you count FSE and ESEC separately).

    Here is an example, which I hope will not be repeated for you but is worth recounting since it was a very serious issue for PASTE 2005. The ESEC/FSE 2005 organizers de-prioritized PASTE, denying it the space and time it needed, delaying approvals for weeks or months (seriously jeopardizing our schedule and publicity), etc. In the end, PASTE 2005 drew more attendance than any other workshop at the conference — this was easy to predict given past attendance at the various workshops, so we were surprised by the uncooperativeness of the ESEC/FSE organizers. (It's impossible to know for certain exactly what caused this, but PASTE bore the brunt of it.)

    On the other hand, PASTE 2007 had no problem with PLDI or FCRC. Also verify that the conference local arrangements chair will take care of the local arrangements, including registration, rooms, etc.
  5. Create a budget, both in order to get sponsorship from the ACM SIGs and as part of your proposal to the conference. The co-located conference will probably set workshop rates. For ACM, you will probably have to fill out a Mini-TMRF.
  6. Optionally, get sponsorship from corporations or other sources. Support on the order of $1,500 is typical and can help with the budget. It's a lot easier for everyone if the organization receiving the funds is in the same country as the sponsoring organization.
  7. Determine the important dates: Think about other conferences when you set submission dates E.g., don't compete directly; permit rejects to be sent to PASTE. Think about hotel, conference registration, and visa deadlines when you set notification dates.
  8. Invite program committee: about 6 people besides the chairs. Really big committees reduce the work for each person, but not by a huge amount for a workshop this size, and they increase work for the chairs. Large committees make it harder to calibrate across reviewers. Large committees make it harder to coordinate a PC meeting. Large committees tend to be correlated (for whatever reason) with lower quality. Set up a mailing list for the PC.
  9. Write a call for papers. It's probably easiest to start from the previous year's CFP. Be sure to indicate that PASTE is not a closed workshop — attendance by non-authors is encouraged. This is important because many closed workshops exist. By this time, you must have decided whether you want both long and short papers.
  10. Create a website. It's probably easiest to start from the previous year's website.
  11. Publicity. Send the call for papers to all the lists and people listed in a separate file that is distributed with this document (perhaps 2-4 months before the submission deadline; I'm not sure what is optimal). Also ask committee members to contact their personal networks. Take advantage of the publicity chair for your co-located conference. Don't forget to send a call for participation as soon as the program is final. Feel free to send out a reminder close to the deadline (but don't over-spam).

Submissions and refereeing

  1. Decide on conference management software. Find out what the conference is using, and you can probably be accommodated under the same contract for free or cheap. SIGPLAN also has a contract covering all SIGPLAN events; check with the SIGPLAN Vice-Chair. There are many free alternatives, as well. Be sure to ask others for their experience with the software. Try to read about, or otherwise learn, the features of the software. PASTE 2005 used Paperdyne, which was fine but didn't have a manual, and in retrospect we did a number of things the hard way that the software could have automated for us, such as assigning reviewers and making the proceedings. PASTE 2007 used START and found it had all the features we wanted.
  2. Near the submission deadline, if there are few submissions, don't panic! For PASTE 2005, as of 24 hours before the deadline, we had only 1 submission. In the last 24 hours we got 37 additional submissions. During the 24 hours before the submission deadline, you will get a lot of questions about format, extensions, etc.; try to be in constant email contact during that time.
  3. Extensions: don't give them. I despise conferences that give extensions. Doing so makes a workshop look unprofessional and ill-organized, it makes it seem that the conference is having trouble attracting submissions, and extensions are inconsiderate to authors who are trying to plan their schedules. So I denied all such requests (there were about 5 or 6, and we had 38 papers submitted overall), and I recommend that future chairs maintain that policy, despite the temptation to get a couple more submissions via an extension. If you do give extensions, do so fairly: be sure to publicize widely so that all authors know about them, and don't give them at the last minute.
  4. You may find my advice about running a PC meeting helpful. I'm sure many other helpful sources of such advice exist; please suggest some!
  5. Ask the PC to suggest invited speakers, and probably decide that at the meeting, too (though afterward is also OK).
  6. When you inform authors that their papers are accepted, remind them that if they need a visa to enter the country in which the workshop is being held, they should start that process early. (It can take 3 months to get a visa to enter the United States, for example.)
  7. Determine the program and schedule. Some of this was determined when you proposed the workshop (1.5 or 2 days?), and some can wait until after the papers are decided (1, 2, or 3 invited talks? 5-minute madness? panel? talks of varying lengths?). Some organizers like keynotes, others not; others add them to fill in if there weren't enough good submissions. The co-located conference may dictate start/end times, coffee breaks, lunch time, etc.
    When making the program, give a title or theme to each session, to emphasize their coherence (or to force you to create such coherence).
    Send the program to the authors; you are likely to have made some typos.

Proceedings

  1. Camera-ready papers: Give authors specific information about the copyright box, so that they don't all come to you asking the same question. The information you give them will look something like this: 
      \conferenceinfo{PASTE}{'05 Lisbon, Portugal}
  \CopyrightYear{2005}
  \crdata{1-59593-239-9/05/0009}
    Ask authors to fill out and sign an ACM copyright form, and fax or mail it to you.
  2. I like to give each paper one extra page in the proceedings, to accommodate changes suggested by the referees. I am very strongly opposed to a policy of giving extra space (or deadline extensions, etc.) only to those who ask. The reason is that the people who don't annoy us — who quietly live by the rules — are the ones that get hurt. I favor a uniform policy — either “no extra pages”, or an openly announced policy of “everyone gets one more page in the proceedings than they got for the submission”. It feels more fair to me.
  3. Proceedings:
  4. Proceedings in Digital Library: ACM SIGSOFT's policy is to offer to publish Workshop Proceedings in the ACM DL, and the abstracts (and a workshop summary) in ACM SIGSOFT SEN. I strongly recommend that you get the proceedings in the DL, with appropriate page numbers. Then — separately and independently — have the abstracts of the papers appear in an issue of ACM SIGSOFT Software Engineering Notes. (You don't want to combine these two steps, because for SEN papers that don't have another place to appear in the DL, the page numbers all start with 1.) Will Tracz can help with both tasks. For SEN, the editor (Will Tracz) needs a List of Papers/Authors (preferably in a certain format) and a list of paper abstracts; he can provide further details. You can also publish in SIGPLAN Notices, but when the 2005 chairs sent mail to the editor at sig.not@acm.org, we never heard back. For PASTE 2007, we did not contact ACM SIGSOFT SEN. Inclusion in the DL was handled by Sheridan.

After the workshop

  1. Consider collecting slides afterward and putting them on the conference webpage. In 2007, we made this optional via email to contact authors and most did send us slides.
  2. The organizers must submit a report to the ACM Vice Chair (SIGPLAN or SIGSOFT) and to the steering committee describing the meeting. The report should include at least the following information: the number of paid attendees the number of papers submitted the number of papers accepted It should also include descriptions of any experiments you tried, or any issues that you had to solve, etc., i.e., anything future meeting organizers might want to know about. It does not need to include financial information. ASCII is fine as a format; it doesn't have to be fancy.
  3. Please improve this manual! We hope it helps you avoid certain pitfalls, but it is certain to be incomplete. Thus, it would be great if you could contribute to future organizers' success in a similar way, by giving them the benefit of your wisdom and experience.

Back to Advice compiled by Michael Ernst.

Michael Ernst