Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions source/about/comparing_PsychoPy_and_EPrime.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
.. _psychopyVersusEPrime:

Comparing PsychoPy with E-Prime
=====================================

If you're looking for an alternative to E-Prime then PsychoPy is an excellent option!

Both applications provide a graphical user interface, where you can add code snippets if you wish, and both will output the entire script for you to view and learn from. E-Prime's script is based on Visual Basic programming language, whereas PsychoPy is built on the more modern Python language.

Both offer millisecond-precise timing and a range of stimuli, although PsychoPy has a slightly wider range and the option for stimulus updates to be every screen refresh.

Another key difference is price. PsychoPy is free and open source, despite having a professional full-time developer team, whereas E-Prime charges $995.00 for a single-user license, and much more if you need hardware plugins.

Flexible dynamic stimulus parameters
------------------------------------

PsychoPy offers a wider range of stimuli: like drifting gratings, “brush” components for drawing inputs, and Random Dot Kinematograms (RDKs), as well as the standard images and movies that E-Prime offers.

PsychoPy also makes it very easy to manipulate stimulus attributes frame-by-frame within its graphical interface. You can simply set almost any parameter to update “on every frame”. If this is possible in E-Prime at all (from looking at the documentation it appears not to be!) then it would certainly be “non-trivial”.

Lab-based studies
------------------

Both PsychoPy and E-Prime were designed for in-lab studies and can interact with a range of hardware including EEG, fMRI and eye tracking devices although, for E-Prime, most of these devices require a plugin and incur an additional $1,995.00 charge for each.

Note also that E-Prime only runs on Windows, whereas PsychoPy runs on Windows, Mac and Linux (or any device with a browser, if you use the JavaScript/Pavlovia output option).

Web-based studies
------------------

PsychoPy experiments can output either Python (for local studies) or JavaScript scripts (for web-based studies). These can run in any device with a modern browser, including Windows, Mac, Linux, iOS, Android, and ChromeOS. When PsychoPy studies run on the web, using Pavlovia.org, you send participants a link to the study, they open it in their browser and the data files are automatically sent back to your Pavlovia account. These web-based studies can also connect seamlessly to Pavlovia Surveys, for Qualtrics-style surveys and questionnaires.

E-Prime is Windows-only and doesn't have a JavaScript output option, so it can't be used for browser-based studies. During the pandemic of 2020, PSTNet did introduce E-Prime Go, which allows E-Prime experiments to be compiled into self-contained apps that can be sent to participants to run on their own machines. Again, the participant must be running Windows and when the study is completed the must follow whatever instructions you give them to send the data back to you. This is a much more cumbersome process than the automatic data collection that Pavlovia provides.

Timing
------

Both E-Prime and PsychoPy have excellent timing, as can be seen in the `Timing megastudy (Bridges et al, 2020)`_.
PsychoPy is also unique in providing `Timing Validator`_ components to allow easy checking of stimulus timing during experiment execution, on relatively cheap hardware such as the Cedrus Riponda button box. This means that, even if your operating system or monitor causes timing issues that the software cannot know about, these can be automatically detected by the PsychoPy using the validator hardware.

.. _Timing Megastudy (Bridges et al, 2020): https://peerj.com/articles/9414/
.. _Timing Validator: https://psychopy.org/builder/components/VisualValidatorRoutine.html#visualvalidatorroutine

Surveys
-------

E-Prime isn't really suited to creating surveys. Although one could manually create each question on its own page, this wouldn't be as powerful as dedicated tools, such as Qualtrics.

PsychoPy's Python library also has somewhat limited built-in support for survey-style studies but, when used in web-based studies, PsychoPy has the added benefit of being able to connect with Pavlovia surveys which are purpose built for this and do have roughly feature parity with Qualtrics. You can create a survey using Pavlovia Survey creator in the browser and even import a Qualtrics .qsf file.

Updates
-------

New versions of PsychoPy are released frequently, with major versions twice per year and bug-fix updates frequently in between. E-Prime, by comparison, has major releases very rarely indeed and its `last bug-fix release was in 2024 <https://support.pstnet.com/hc/en-us/articles/360009483234-RELEASE-INFO-E-Prime-3-0-Change-History-27275>`__ although there is a `list of known issues <https://support.pstnet.com/hc/en-us/articles/360009608053-RELEASE-INFO-Known-Issues-in-E-Prime-3-0-27789>`__.

Pricing and vendor lock-in
--------------------------

E-Prime costs roughly $995 per concurrent license, and adds a further $1,995.00 if you want to use a plug-in for EEG or eye-tracking extensions (for each extension). Its experiment file formats are proprietary and its programming language (Visual Basic) is no longer widely used in any other context.

PsychoPy is free and open-source software. Running studies online using Pavlovia does incur a small charge but even that is small compared to standard lab use of E-Prime (£0.26 per respondent or £2000 for an unlimited institutional licence). PsychoPy also offers paid consultancy and training options. PsychoPy uses the Python language which is widely used in many other contexts, and its experiment file formats are open and human-readable (XML-based and with a public format description).

Community
---------

E-Prime is historically the more widely-used option, having been very much the dominant package in the early 2000s, when PsychoPy was just being conceived.

While it isn't clear how many users E-Prime still has now, `PsychoPy has over 50,000 monthly active users <https://usage.psychopy.org>`_ and a very active `users forum <https://discourse.psychopy.org/>`_ with over 15,000 members.
43 changes: 43 additions & 0 deletions source/about/comparing_PsychoPy_and_Gorilla.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
.. _psychopyVersusGorilla:

Comparing PsychoPy with Gorilla
=====================================

Gorilla is a comparable experiment builder, in some ways, to PsychoPy. It's very easy to use for simple experiments. It has the added convenience that it can be used entirely online without installing any software, but it is less suitable for lab-based studies, especially where millisecond-precise timing, or external hardware, are needed. Gorilla is also considerably more expensive.

PsychoPy provides the option to run the same study locally in the lab using Python, or online using JavaScript.

Flexible dynamic parameters
---------------------------

PsychoPy allows you to control every aspect of every stimulus. For instance, while Gorilla and PsychoPy both support images as stimuli, in PsychoPy you can rotate, flip, stretch your image, or even set its transparency or contrast, all without writing a line of code. None of those are possible in Gorilla's Image Node settings.

Not only do PsychoPy stimuli support a very rich set of parameters, but these can be updated on every trial or even every screen refresh! That means stimuli can be made to move, resize, bounce, or alter their transparency constantly, making for more dynamic interactive experiments.

Lab-based studies
-----------------

Because PsychoPy is a Python-based desktop app, you can connect with external devices such as EEG, eye trackers, fMRI and photometers relatively easily (anything possible with Python code can be done in PsychoPy). Because Gorilla is a browser based tool it cannot directly communicate with external devices like serial ports without an additional bridge (e.g. a local application that connects the web browser with the hardware device via websockets).

Conversely PsychoPy can generate either a web-based JavaScript study, which has similar limitations to Gorilla, it can also generate a Python script that runs entirely locally and can easily be connected to devices such as eyetrackers (including Tobii, SR Research, Pupil Labs, EyeLogic devices), brain imaging equipment (such as EEG, fMRI, and fNIRS studies), physiology hardware (such as BIOPAC devices) and calibration devices such as photometers.

The timing of Gorilla (and any web-based experiments, including when you use PsychoPy in web-based studies via Pavlovia) is also less precise. Gorilla staff sometimes claim “millisecond precision” but while most JavaScript packages use a millisecond precise timer that does not equate with millisecond-precise timing. The critical issue is that web-based packages cannot very precisely detect when a screen refresh occurred or when a sound actually reached the speakers. If you measure a reaction time but don't really know when the stimulus started, then it doesn't matter how precise your timer is. For analogy, consider measuring a sprint race with a stopwatch. The stopwatch may have incredible precision, but if you hit the button with your thumb the precision is lost. See the `Timing Megastudy`_ for details of what is actually possible with online and desktop packages.

.. _Timing Megastudy: https://peerj.com/articles/9414/


Surveys
-------

Both PsychoPy and Gorilla have solutions to present surveys and questionnaires online. For PsychoPy the easiest approach is to use a Pavlovia Survey. Both Pavlovia Surveys and Gorillas questionnaire builder support a range of question types including free-text, rating scales, multiple choice grids and rankings. Pavlovia Surveys offer some question types not available in Gorilla Questionnaire builder such as dynamic matrices, signature responses and file uploads. Both platforms offer the option to directly import a .qsf file from Qualtrics.

For local PsychoPy experiments (not using Pavlovia surveys) there is a form component, but the range of question types here are more limited. You can still use type responses, radio choice options and sliders.

Pricing
-------

PsychoPy is completely free and always will be. That means you can build an experiment, run it on your local machine and gather data at no cost.

If you want to run a study _online_ you can host the study using Pavlovia.org. Pavlovia is a paid-for service, but very low cost (£0.26 per participant or £2000 for a whole institution with unlimited use). By comparison, with Gorilla you can build your experiment for free, but you can't run it and collect data without credits or a licence (£1.09 per participant or options ranging up to £11340 for a department licence with unlimited use - see this page for updates on Gorilla's pricing).

Both Pavlovia and Gorilla teams can provide training via onboarding sessions and private webinars. To buy the unlimited Pavlovia license including private webinars costs £5,200 compared with £20,470 for Gorilla.
55 changes: 55 additions & 0 deletions source/about/comparing_PsychoPy_and_OpenSesame.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
.. _psychopyVersusPsychtoolbox:

Comparing PsychoPy with Open Sesame
=====================================

PsychoPy and Open Sesame are both python based. In fact, Open Sesame uses PsychoPy as one of its “backend” options and the Open Sesame documentation recommends the PsychoPy backend if good timing is required, but this is still _not_ as good as PsychoPy itself, as explained below.

Both applications provide a graphical user interface, where you can add code snippets if you wish. PsychoPy provides the option to view the code that the Builder interface generates and to write experiments by hand using Python using the “Coder” interface, if desired, neither of which are possible in Open Sesame.

Flexible dynamic parameters
----------------------------

PsychoPy and Open Sesame differ in their focus for the user, PsychoPy is very temporal in nature (everything follows a time line or “flow”) whereas in Open Sesame the user visualises their experiment more spatially similar to a set of slides.

Because of the temporal focus of the PsychoPy builder interface, it is very easy to manipulate stimulus attributes frame-by-frame via drop-down with no code needed. You can simply set almost any parameter to update “on every frame”. In Open Sesame, setting stimuli to be dynamic is not built-in but requires the addition of a “coroutines” plugin and has a more limited range of uses.

Lab-based studies
------------------

Both PsychoPy and Open Sesame were designed firstly for in-lab studies and can therefore interact with a range of hardware including EEG, fMRI and eye tracking devices.

Timing
------

PsychoPy has the more precise timing, as can be seen in the `Timing Megastudy (Bridges et al, 2020)`_. Although Open Sesame is using the PsychoPy library as its recommended backend, it uses outdated functions within that library that have been superseded for timing reasons. As a result the keyboard timing and audio stimulus presentation, in particular, have notably poorer performance than PsychoPy itself and that can be seen in the above measurements.

PsychoPy is also unique in providing `Timing Validator`_ components to allow easy checking of stimulus timing during experiment execution, on relatively cheap hardware such as the Cedrus Riponda button box. This means that, even if your operating system or monitor causes timing issues that the software cannot know about, these can be automatically detected by the PsychoPy using the validator hardware.

Surveys
------------------

It is possible to present survey-style questions in Open Sesame, much like you would in PsychoPy, through presenting text and different response types (e.g. rating scales and sliders).

PsychoPy has the added benefit of Pavlovia surveys which run on survey.js. You can create a survey using Pavlovia survey creator in browser (no need to install software) and you can even import a Qualtrics .qsf file. This is not possible with Open Sesame.

Pricing
------------------

Both PsychoPy and Open Sesame are free for building and running studies. For in-lab studies they won't cost you anything.

If you want to run a study online then PsychoPy offers a hosting service, pavlovia.org (costing £0.26 per respondent or £2000 for an unlimited institutional licence). Open Sesame has the option of JATOS or MindProbe servers, which are free but don't offer the security certification that Pavlovia does. Both PsychoPy and Open Sesame offer paid consultancy support for users needing help creating their experiments.

Community
------------------

Both platforms also have community user forums and benefit from communities of enthusiastic open-source users.

PsychoPy has a `larger developer/contributor team <https://github.com/psychopy/psychopy/graphs/contributors?from=18%2F04%2F2026>`__, a larger more active user community, and a proven business model where Open Science Tools is supported by related services (hosting of Pavlovia studies and provision of consultancy services) which is how PsychoPy itself remains free while employing several full-time developers.

Open Sesame has seen much less recent development and is `almost entirely written by Sebastiaan Mathôt <https://github.com/open-cogsci/OpenSesame/graphs/contributors?from=20%2F07%2F2024>`__. It has a “Bus Factor” of roughly “1” as a result, meaning that if Sebastiaan stops work on it he project will likely not be maintained any further.

Both packages have a user forum. `PsychoPy's forum <https://discourse.psychopy.org/>`__ has over 15,000 members and is very active, with 5-10 posts per day, whereas the `Open Sesame forum <https://forum.cogsci.nl/categories/opensesame>`__ typically has around 5 posts per month.

.. _Timing Megastudy (Bridges et al, 2020): https://peerj.com/articles/9414/
.. _Timing Validator: https://psychopy.org/builder/components/VisualValidatorRoutine.html#visualvalidatorroutine
Loading
Loading