THEMIS Web and Data Server Acceptable Use Policy
Purpose and Background
The THEMIS web site and data server are maintained by the THEMIS Science Operations Center (SOC) as a service to the heliophysics research community and the public. Our highest priority is to preserve efficient, reliable access to mission documentation, software tools, and scientific data products.
In recent years, THEMIS and many other scientific data providers have experienced dramatic increases in automated web traffic. We routinely observe sustained periods of hundreds of requests per second, originating from thousands of distinct IP addresses distributed across numerous networks and geographic regions.
Although individual clients may generate relatively few requests, their combined activity can overwhelm server, filesystem, and network resources. This can cause severe performance degradation, connection timeouts, and interruptions of legitimate scientific data access.
Some of this activity appears to involve coordinated, large-scale crawling through distributed proxy networks. The identities and purposes of the operators are not always known.
To protect the availability of our services, we have found it necessary to implement increasingly restrictive measures against excessive automated traffic. Unfortunately, these measures may occasionally affect legitimate researchers and institutions.
If you believe your legitimate access has been blocked in error, please contact [mailto:themis-webadmin@ssl.berkeley.edu]. Include your IP address, the approximate time of the problem, the resources you were trying to access, and the software or method you were using. We will work with you to restore appropriate access.
Acceptable Uses of THEMIS Web-Facing Resources
1. Web browsing, documentation, and software downloads
We welcome normal browsing of THEMIS web pages, scientific documentation, tutorials, and software resources.
Reasonable automated indexing by search engines, AI/LLM crawlers, and similar services is also permitted, provided it does not interfere with normal operation of the site.
Automated clients are expected to use conservative request rates, avoid unnecessary concurrent connections, and respect published access restrictions.
We reserve the right to restrict or block automated activity that consumes excessive server resources or impairs access for other users.
2. Scientific use of THEMIS data products
We support and encourage the use of THEMIS data products for scientific research, education, and related activities.
This includes routine data discovery and downloads using IDL SPEDAS, PySPEDAS, Autoplot, and other scientific analysis tools, as well as direct downloads through standard web browsers and HTTP clients.
Researchers may retrieve data appropriate to their scientific investigations without prior authorization, provided their usage does not place an unreasonable burden on the data server.
3. Large-scale downloads, mirroring, and archival projects
Large-scale downloading, systematic archival, or mirroring of substantial portions of the THEMIS data archive must be coordinated in advance with the THEMIS Science Operations Center.
Such arrangements help ensure that data transfers are conducted efficiently and do not interfere with access by other users.
For example, NASA's Space Physics Data Facility (SPDF) acquires THEMIS data products for distribution through CDAWeb. UCLA and other institutions maintain additional THEMIS data mirrors. These activities are coordinated with the THEMIS SOC using agreed-upon access methods and schedules.
We welcome similar collaborations with other research institutions and data providers, and can discuss appropriate synchronization methods, scheduling, and data access arrangements.
Prohibited Uses
The following activities are prohibited without prior authorization:
1. Uncoordinated bulk harvesting or mirroring. Systematically downloading substantial portions of the THEMIS data archive without coordination with the THEMIS SOC, particularly when doing so generates excessive traffic or resource consumption.
2. Circumvention of access restrictions. Using rotating IP addresses, distributed proxy networks, misleading client identification, or other methods to evade rate limits, network blocks, or other access controls. Ordinary use of institutional proxies and VPNs is not prohibited.
3. Disruptive automated access. Generating excessive request rates, concurrent connections, repeated directory traversals, redundant downloads, or other activity that degrades the availability or performance of our services.
4. Interference with service availability. Any deliberate or avoidable activity that overwhelms server, filesystem, or network resources, or otherwise prevents legitimate users from accessing THEMIS resources.
Enforcement and Exceptions
To maintain the availability and reliability of THEMIS web-facing services, the THEMIS Science Operations Center reserves the right to restrict or block access when activity violates this policy or threatens the operation of our systems.
Restrictions may be implemented without advance notice when necessary to protect service availability.
We recognize that legitimate scientific workflows may occasionally be affected. Users experiencing access problems are encouraged to contact the THEMIS Science Operations Center, so that we can investigate and work toward restoring appropriate access. If your access is interrupted, or if you anticipate a project involving substantial automated data retrieval, please contact:
THEMIS Science Operations Center [mailto:themis-webadmin@ssl.berkeley.edu]
We are happy to work with researchers and institutions to develop sustainable access arrangements.
Our goal is not to limit the scientific use of THEMIS data, but to ensure that these resources remain reliably available to the entire community.