Hindawi

Complexity
Volume 2022, Article ID 6860915, 9 pages
https://doi.org/10.1155/2022/6860915

Research Article
Quick Compression and Transmission of Meteorological Big
Data in Complicated Visualization Systems

He-Ping Yang,1 Ying-Rui Sun ,1 Nan Chen,1 Xiao-Wei Jiang,1 Jing-Hua Chen,1
Ming Yang,2 Qi Wang,1 Zi-Mo Huo,1 and Ming-Nong Feng1
1
National Meteorological Data Center, Beijing, China
2
Zhejiang Meteorological Information and Network Center, Zhejiang Meteorological Bureau, Hangzhou, China

Correspondence should be addressed to Ying-Rui Sun; [email protected]

Received 24 November 2021; Revised 12 January 2022; Accepted 12 March 2022; Published 5 May 2022

Academic Editor: Ning Cai

Copyright © 2022 He-Ping Yang et al. This is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
The sizes of individual data files have steadily increased along with rising demand for customized services, leading to issues such as
low efficiency of web-based geographical information system (WebGIS)-based data compression, transmission, and rendering for
rich Internet applications (RIAs) in complicated visualization systems. In this article, a WebGIS-based technical solution for the
efficient transmission and visualization of meteorological big data is proposed. Based on open-source technology such as HTML5
and Mapbox GL, the proposed scheme considers distributed data compression and transmission on the server side as well as
distributed requests and page rendering on the browser side. A high-low 8-bit compression method is developed for compressing
a 100 megabyte (MB) file into a megabyte-scale file, with a compression ratio of approximately 90%, and the recovered data are
accurate to two decimal places. Another part of the scheme combines pyramid tile cutting, concurrent domain name request
processing, and texture rendering. Experimental results indicate that with this scheme, grid files of up to 100 MB can be
transferred and displayed in milliseconds, and multiterminal service applications can be supported by building a grid data
visualization mode for big data and technology centers, which may serve as a reference for other industries.

1. Introduction Forecast System (GFS) model [4], the Global Regional

Assimilation and PreEdiction System Global Forecast
Currently, the development of information collection and System (GRAPES_GFS) model of the China Meteorological
storage technology has ushered in the era of big data in Association (CMA) [5], and the real-time High-Resolution
various industries, as the amounts of data being recorded, CMA Land Data Assimilation System (HRCLDAS) [6]. The
processed, and analyzed have exploded. In particular, most common storage formats for such grid data products
meteorological data are among the most important types of are General Regularly-distributed Information in Binary
data encountered in people’s daily lives, playing an es- form (GRIB) and Network Common Data Form (NetCDF).
sential role in understanding the environment, natural The former is a file format that was designed by the World
resources, the economy, and other aspects of life [1]. To Meteorological Organization (WMO) for storing and
address society’s need for refined meteorological data, grid transmitting meteorological grid data, such as the outputs
data products for observations and predictions based on of numerical weather prediction models; this format is
radar data, satellite data, and station observations have concise enough to be widely used in meteorology to store
been extensively utilized [2]. On a global scale, the available historical and forecast weather data [7]. The latter is an
meteorological grid data mainly include numerical fore- array-oriented and network sharing-based data description
casting products from the European Centre for Medium- and coding standard proposed by scientists of the Unidata
Range Weather Forecasts (ECMWF) [3], the National project at the University Corporation for Atmospheric
Centers for Environmental Prediction (NCEP) Global Research (UCAR) [8].
2 Complexity

Files in these commonly utilized grid data formats can However, the local installation process has relatively high
only be opened by professional applications (apps) and can system requirements and is therefore not suitable for public
be used to obtain values at specific locations or to analyze the services.
spatial distributions of variables such as temperature or As mobile terminal apps such as Wireless Application
rainfall [9]. For meteorological big data, which tend to have Protocol (WAP) browsers and WeChat have been developed
strong geographical spatial characteristics and location for IOS and Android operating systems, rich Internet ap-
correlations, geographical information system (GIS) tech- plications (RIAs), which are web-based applications
nology is usually combined with tools for visual expression designed to deliver the same features and functions normally
and application provided by common services [10]. Panoply associated with desktop applications, have become essential
from the National Aeronautics and Space Administration platforms that can run in web browsers without installation.
(NASA) Goddard Institute for Space Studies, which is a One of the earliest attempts to make RIAs accessible was to
viewing tool rather than a data extraction tool that supports use the World Wide Web Consortium (W3C) standard for
multiple formats such as GRIB, requires the Java Runtime Accessible Rich Internet Applications (ARIA) [18]. This
Environment [11]. MeteoInfo (i.e., MeteoInfoMap and technology has been used to visualize meteorological big
MeteoInfoLab), which was developed by the Chinese data with web-based GIS (WebGIS) tools. Rain Viewer, an
Academy of Meteorological Sciences, is an integrated all-in-one weather radar and rain forecast app for predicting
framework for both GIS applications and scientific com- storm tracks, is available for 90 countries and offers the most
putation environments that is utilized by the meteorological comprehensive weather radar coverage on the market,
community to visualize and analyze spatial and meteoro- displaying a single map with data from 1000+ Doppler
logical big data in multiple data formats [12]. As the cen- radars with the option of viewing information about each
terpiece of National Weather Service operations in China, radar on the map. To provide this functionality, all requests
the Meteorological Information Comprehensive Analysis to Rain Viewer are routed through an online web service that
and Process System (MICAPS) is a complicated computer overlays the Rain Viewer data on a map tile with 256 or 512
system that combines meteorological, satellite, and radar pixels centered on the user’s current location and then
data into one workstation and allows graphical and alpha- resizes the image to match the screen size of the device [19].
numeric weather data in GRIB format to be read, analyze, In the WebGIS service OpenStreetMap, the layer overlay is
combined, and manipulated [13]. displayed within milliseconds by using leaflet technology
All of the above methods have been used to display local and considering the aging of single-slice requests. Thus, this
meteorological files. However, with the emergence and service cannot meet the requirements of real-time interac-
popularization of cloud storage, new types of applications tions, such as changing the color range or filtering by value.
have arisen in which computing resources are no longer Moreover, the layer is virtualized, and the user experience
localized but rather distributed, heterogeneous, and dy- can be poor if the maximum image resolution is exceeded
namic. The Grid Analysis and Display System (GrADS) is a after the image is enlarged. Advancements in vector tile
widely used drawing software tool in meteorology. It has two technology have offered solutions to the above problems, in
main functions, namely, data processing and image display, which vector tile layers are saved as compressed files in the
and plays a role in the meteorological research community Protocolbuffer Binary Format (PBF) file format [20]. Such a
similar to that of the World Wide Web in facilitating in- compressed file, which contains vector map data in one or
formation exchange over the Internet. GrADS has unique more layers, can be rendered and styled based on the style of
characteristics, mainly for scientific research and business each layer. The data in a vector tile include geographic
personnel involved in atmospheric and marine research. features in the forms of points, lines, and polygons [21]. For
With its powerful data analysis capabilities, flexible envi- weather radar, this solution goes beyond simply resampling
ronment setup, wide range of mapping types and variety of the data and aims to generate vector contours based on the
map projection methods, GrADS has greatly aided meteo- raw radar data. With the data in the form of vector polygons,
rological research [14]. The Integrated Data Viewer (IDV) the AerisWeather Mapping Platform (AMP) can render
from Unidata/UCAR is a Java-based software platform for radar data at any zoom level without reducing the resolution
analyzing and visualizing geoscience data [15]. The IDV or quality. This also allows the user to control how much
combines the abilities to display and analyze satellite im- smoothing is applied to the radar data, enabling clean and
agery, gridded data (such as numerical weather prediction smooth radar imaging at the city and neighborhood levels
model outputs), GIS data, and other data in a single interface [22].
[16]. It has been integrated with common scientific data Meteorological big data vary spatially and temporally,
servers, including Unidata’s THematic Realtime Environ- and any dynamic vector slicing scheme must include data
mental Data Distributed Services (THREDDS) Data Server preparation, slicing, and front-end visualization, with high-
(TDS) [17], as data sources to enable easy access to a large performance requirements for the server hosting the spatial
number of real-time and archival datasets. The IDV is the database (PostGIS) [23]. Hence, complex visualization
main tool used in the computer laboratory portion of various technology for meteorological big data is gradually devel-
meteorological courses at colleges and universities. oping in the direction of data file compression prior to
These localized applications can read files directly from a transmission, followed by foreground decoding. The Null
local disk, efficiently download them to a local disk, or school designed a global visual display system (Nullschool.
integrate them with common scientific data servers. net) [24] for ECMWF forecast data, which converts a map
Complexity 3

from the Natural Earth dataset into the TopoJSON format to on the HTML5 and Vue frameworks, the front-end uses
serve as the base map, utilizes the EPAK format to transform Mapbox GL [29, 30] technology to satisfy the demands of
and compress the grid data, and applies Node.js to rapidly dynamic meteorological big data visualization services while
render and display the foreground [25]. Since the launch of considering compression, slicing, display, and other factors.
this system, the Tokyo Meteorological Bureau and other The proposed scheme offers a mid-platform support mode
institutions have developed the Tokyo Wind Map based on for visualizing meteorological grid data that have been
this technical framework, which has been extensively pro- published in the meteorological visualization column of the
moted. Although this framework makes full use of the China Meteorological Data Service Center, via a mobile app,
advantages of rapid visualization on a web terminal, there or on WeChat. This scheme provides a fast and convenient
are still incompatibility problems on mobile terminals for solution for rapidly visualizing and rendering grid data and
base maps with coastline contour resolutions of 50 km or can serve as a reference for grid data visualization appli-
110 km in the JSON format; hence, it is difficult to use this cations in other industries.
framework for fine-scale service applications. Lytvyn et al.
[26] and Prastika et al. [27] have implemented multichannel
support applications such as PC browser, WAP, and mobile
2. Methodology
applications based on the OpenStreetMap online map by The proposed system includes big data processing, trans-
integrating slices with data compression and transmitting mission, and page rendering and involves technologies such
only single slices within 30 kB for faster transmission and as data analysis, compression, browser transmission, page
visualization. For data visualization based on the browser/ data restoration and splicing, as well as WebGL [31] ren-
server (B/S) architecture, the minimum resolution of me- dering. The detailed design is shown in Figure 1.
teorological grid data reaches more than 9 km worldwide, Data processing and compression: Red-green-blue
and the file size for a single transmission is between 700 kB (RGB) channels are used to compress and store the data in
and 2 MB. A single request can reach a second-level re- high-low 8-bit PNG files to maintain data accuracy to the
sponse, and dynamic rendering allows updating, which fullest extent possible during the transmission process.
satisfies the requirements for large-scale services. However, Data slicing and transmission: Based on pyramid slicing
various disaster prevention and emergency mitigation technology, slicing is performed with a specified minimum
support applications need more refined grid data services. In scaling resolution, and distributed multithreading is used
2021, the CMA released the HRCLDAS product [28], which during transmission to increase the timeliness of trans-
covers East Asia with a resolution of 1 km. The grid size is mitting the data from the server to the page terminal.
7000 ∗ 4500, and the data volume of a single file reaches Data visualization rendering: The browser obtains slices
106 MB. The main applications for visualizing such data and performs slice stitching, while the Mapbox GL com-
superimpose a transparent image directly on top of the map ponent based on WebGL technology realizes fast dynamic
and reduce the zoom or resolution layer by layer, resulting in rendering.
a loss of eigenvalues (e.g., the maximum and minimum
values) and thus affecting the front-end rendering results.
Therefore, it is extremely difficult to balance efficiency and 2.1. Data Processing and Compression. First, the source GRIB
data accuracy in page rendering based on the B/S file is transformed into a float array by PYGRID [32] in
architecture. Python and is generally stored as 4-byte data with a max-
In conclusion, visualization systems based on the B/S imum legend display resolution of 0.1. The data can be
architecture are the most user-friendly option; however, retained to 1 significant digit before being stored, that is, the
their compression and transmission methods have become original value O is multiplied by 10 and rounded to obtain
increasingly complicated as the demand for refined services the integer C. The image is saved as an RGB compressed
and individual file sizes have increased. Compression is a image with 2 bytes in the G and B channels in the range of
highly efficient method for files smaller than 10 MB, but the [− 32768, 32767]. To minimize data loss, the image is stored
visualization process achieves better transmission and in the PNG format by adopting an LZ77-derived algorithm
rendering pressure when the file size is approximately for file compression, thus resulting in a small data volume, a
100 MB or larger, and 100 MB is a common file size for high compression ratio, and no data loss.
refined meteorological services. Therefore, there is a need to
design a fast transmission and display scheme for meteo- I11 · · · I1W
⎡⎢⎢⎢ ⎤⎥⎥⎥
rological grid data on PCs, mobile browsers, WAP apps, O � ⎢⎢⎣ ⋮ ⋱ ⋮ ⎥⎥⎥⎥⎦;
⎢
⎢
WeChat applets, and other platforms based on an open- (1)
source WebGIS service platform. IH1 · · · IHW
In this article, we propose a customized scheme for use in C � round(O × 10).
complicated visualization systems for meteorological big
data. At the back end, a high-low 8-bit compression algo- The high 8-bit and low 8-bit values of the converted data
rithm is adopted, and customized slice transmission is re- C are stored in the G and B channels, respectively. The
quired to ensure high network transmission efficiency. Based specific operations are shown as follows:
4 Complexity

PYGRID Data
It is assumed that if the source image G has a size of
GRIB data decoding M × N and the scaled target image g has a size of m × n, the
Data processing and coordinates of g on G can be calculated using formula (3).
compression
M
x �x× ,
Compress and store
′ m
with high-low 8-bits (3)
G N[: ][: ][:, :, 1] � U;

G N[: ][: ][:, :, 2] � D.

Pyramid slices in PNG format
Data slicing and
transmission
As shown in Figure 2, (x′ , y′ ) denotes the location of a
point P′ in the original image, which corresponds to the
Concurrent requests on position g(x, y) in the compressed image; the value at this
the web
point is obtained by interpolating from the pixel values at the
16 neighborhood points (P00, . . ., P33). If the position of
Acquire and P11 is (x, y), then the position of P′ can be expressed as
splice (x + u, y + v), where u and v represent the fractional parts of
Data visualization
rendering
the pixel coordinates.
Once the influence weights of the 16 neighborhood
Visualization points relative to point P’ have been calculated, the value of
P’ can be obtained and mapped to the scaled image g. The
Figure 1: Design of the system. basic function for bicubic interpolation is shown in Formula
(2), where a � − 0.5:
C ⎪
⎧ (a + 2)|x|3 − (a + 3)|x|3 + 1, for |x| ≤ 1,
U � trunc􏼒 􏼓; ⎪
⎪
256 ⎨
W(x) � ⎪ a|x|3 − 5a|x|2 + 8a|x| − 4a, for 1 <|x| < 2,
⎪
⎪
D � C%256, ⎩
0, otherwise.
􏼂R11 , G11 , B11 􏼃 · · · 􏼂R1W , G1W , B1W 􏼃 (4)
⎢
⎡
⎢
⎢ ⎤⎥⎥⎥
⎢
⎢ ⎥⎥⎥ (2) When the rows and columns are separated, the distance
G N �⎢
⎢
⎢
⎢
⎢ ⋮ ⋮ ⋮ ⎥⎥⎥,
⎥⎥⎥ between the pixel value to be calculated and the known pixel
⎢
⎢
⎣ ⎦ value P00 in Figure 2 can be expressed as (1 + u, 1 + v); hence,
􏼂RH1 , GH1 , BH1 􏼃 · · · 􏼂RHW , GHW , BHW 􏼃 the abscissa-coordinate weight of the P00 pair is W(1 + U)
and the ordinate weight is W(1 + V), yielding a corre-
G N [: ][: ][:, :, 1] � U; sponding value contribution of
Pix00 × W(1 + u) × W(1 + v). The contributions from the
G N [: ][: ][:, :, 2] � D. other 15 points can be calculated similarly. Finally, the pixel
value at the point in the map scaled to the image G can be
where U and D represent the intermediate values, while G_N calculated using formula (5).
denotes the newly created PNG image used to store the 3 3
corresponding compressed values. G(x, y) � 􏽘 􏽘 Pixij × W(i) × W(j). (5)
To improve the efficiency of page data transmission, the i�0 j�0
data are processed in accordance with the image pyramid
model. The compressed PNG file is scaled down using the Notably, a browser can process only a limited number of
image pyramid approach with bicubic interpolation. The concurrent requests for the same domain name, which re-
tilemap pyramid model is a multiresolution hierarchical stricts the number of simultaneous requests that can be
model. The resolution decreases from the bottom to the top served during page rendering, resulting in requests queuing
of the tile pyramid; however, the geographical range of the or timing out. This process occurs on GIS service websites
representation remains constant. The image is first scaled such as Google and Baidu Maps, which add subdomains and
and then filled with squares in accordance with the tilemap domain dashes to increase the number of concurrent re-
pyramid model. quests that can be served [34]. However, given the increased
difficulty of DNS resolution for an excessive number of
domain names, the concurrency of each secondary domain
2.2. Data Slicing and Transmission. The original image serves name should be limited to 2–4.
as layer 0 of the pyramid and is scaled by 2×, 4×, 8×, and 16× The scheme proposed here, which is based on a B/S
via bicubic interpolation [33]. In numerical analysis, bicubic service framework, uses an Nginx server, which is a light-
interpolation is the most commonly applied interpolation weight Web server/reverse proxy server, and an e-mail
method in two-dimensional space. (IMAP/POP3) proxy server distributed under a BSD-like
Complexity 5

P00 P01 P02 P03 website CSS, JS, XML, and HTML files during transmission,
thus boosting the access speed and optimizing Nginx
performance.

P10 P11 P12 P13

2.3. Data Visualization Rendering. During the rendering
v process, the number of slices should be adjusted to account
u
for different screen resolutions and map magnifications. The
P’
coordinates of the map are translated into screen coordi-
P20 P21 P22 P23 nates to realize the transformation between the longitude
and latitude coordinates of the map and the screen coor-
dinates. Once the scale problem has been resolved, the
spatial information of the latitude and longitude ranges on
P30 P31 P32 P33 the earth corresponding to the GRIB file is transformed to
the range of the screen. Based on the size of each layer of the
Figure 2: Bicubic interpolation diagram.
image pyramid in the GRIB file, the pyramid slice that offers
the best rendering is finally selected (Figure 3).
protocol, as proxies, thus occupying less memory and en- The distance between any two points expressed in terms
abling high concurrency. Moreover, Nginx supports the gzip of longitude and latitude can be calculated using the fol-
compression function, which can be used to compress the lowing flow formula:

cos(lats) ∗ cos(late) ∗ cos(lons − lone)

Distance((lons, lats), (lone, late)) � R ∗ cos− 1 􏼢 􏼣, (6)
+sin(lats) ∗ sin(late)

where R is the radius of the earth, which is approximately where tilex is the upper left corner of the data in the x
equal to 6371.0 km. direction, disx denotes the distance between the upper left
The corresponding screen distance can be calculated corner of the screen and the upper left corner of the data in
from the Mapbox scale. the x direction, colstart represents the index of the starting tile
column, colend represents the index of the ending tile row,
screenD � Distance((lons, lats), (lone, late)) ∗ scale. (7)
interval denotes the length and width of a tile, and ceil
Finally, the layer number of the selected slices can be denotes the operation of rounding up. Similarly, the row and
determined based on the diagonal width of each layer image column indices of the starting and ending tiles can be ob-
in the pyramid. tained. Because the tile coordinate range is greater than the
Here, orgSZ denotes the diagonal image size in the screen coordinate range, all tiles need to be offset. Starting
compressed resource file (Figure 4). The result of dividing with the upper left corner of the screen, the position offsets
“orgSZ” by “screenD” indicates how many times larger the in the CSS file can be obtained by calculating the difference
visible range on the screen is than the size of the compressed between the pixel coordinates of the upper left corner of each
file (layer 0). Because consecutive layers differ in size by a tile and those of the upper left corner of the screen [35].
factor of 2, the indicated layer is determined by rounding up Based on the Vue development framework, the proposed
the value to a power of 2. scheme comprehensively considers the spatiotemporal at-
tributes of meteorological big data and the demands of fast
􏽲������� rendering, utilizes Mapbox to support geographical infor-
2 orgSZ
Pyramid layer � . (8) mation services, and applies WebGL high-performance
screenD
front-end rendering technology for data visualization. Vue is
Thereafter, the latitude and longitude ranges on the a progressive and high-performance JavaScript framework
screen are obtained, namely, the upper left (lux , luy ) and for front-end page display that uses view layer rendering as
lower right corner (rdx , rdy ), and the tile index is obtained its core. Vue utilizes a component mechanism, a routing
in accordance with the flow progress. mechanism, and a state management mechanism to quickly
realize front-end high-frequency Document Object Model
disx � tilex − lux , (DOM) operations and efficient page interactions. Due to
disx the use of the NodeJS service and the utilization of Node
colstart � ceil􏼠 􏼡, if disx > 0 else � 0, Package Manager (NPM) to install the Vue command-line
interval (9) interface (Vue CLI), the framework can be built quickly.
rdx − lux Mapbox, which has corresponding GIS engines for different
colend � colstart + ceil􏼠 􏼡, platforms (e.g., PC and mobile), is an efficient WebGIS
interval
development framework. As a Mapbox component [36],
6 Complexity

Get map zoom level=ZS

Get the projection range of the map

Get the slice level index corresponding to the size of the area
by ZS visible on the screen (Lons,lats)-
(lone,late)

Get the slice indices

Merge and splice slice data

Render visualizations

Figure 3: The rendering process.

Mapbox GL has been used for HTML5 web development. (3500 ∗ 3500), layer 2 (1750 ∗ 1750), layer 3 (875 ∗ 875), and
Mapbox GL is a JavaScript library that can render a large layer 4 (437 ∗ 437) were obtained.
number of map elements while allowing for smooth in- Table 1 shows the minimum and maximum compression
teractions and animation. ratios of the five data layers, which reached 30 and 95, re-
spectively, after PNG compression and conversion. The
average file size of the 24 experimental files was calculated.
3. Experiments and Analysis
The degree value equivalent to the pixel interval corre-
To examine the performance and effectiveness of the whole sponding to each scale ratio in the five layers of the pyramid
proposed process, high-low 8-bit compression was devel- was determined based on the scaling coefficient. The pixel
oped in Python 3.7, and the grid data (GRIB2) were decoded size in layer 4 was taken as the size of a single tile to slice the
into arrays by PYGRID. The web page was developed in data from the other four layers. The maximum size of a
Visual Studio XCODE, and the web server was deployed single-slice file was less than 90.7 kB, sufficient to guarantee
using Nginx 1.16.0. The progress server and web server were fast transmission. When the sizes of the slice files were
64-bit Linux servers, and the CPU for the experiment was an compared, it was found that the slice files tended to be larger
eight-core Intel Core i5 @ 2.30 GHz with 16 GB of memory. in areas with rainfall due to the data distribution in these
locations.

3.1. Data Processing. The data used in the experiments were

3.2. Transmission Efficiency. In the experiments, the time it
obtained from the CMA Multisource Precipitation Analysis
took for all slices in the compressed file (layer 0) to be
System (CMPAS) [37], which are available through the
transmitted from the server to the browser was a thousand
China Meteorological Data Service Center (http://data.cma.
times faster than the time it took for the original file to be
cn). The data include latitude and longitude ranges of
transmitted when 256 concurrent channels were used. The
70–140° E and 15–60° N. The experiments included 24 hourly
compressed PNG transmission efficiency is compared be-
precipitation fusion products with a resolution of 1 km ∗ tween the full-size image and its slices (Figure 5).
1 km from 00 : 00 to 23 : 00 on July 20, 2021; these data were The maximum amount of data that could be requested by
chosen as an example to evaluate and compare the data
the browser includes the map and data slices. In these ex-
processing with the foreground display. The data included periments, the actual maximum amount requested was less
7000 latitude points and 4500 longitude points, with over than 60 because the visible slices were determined according
300 million data points in a single file, and the single file size to the range of the map. Sixty-four concurrent channels with
of the hourly precipitation fusion product was 101.3 MB. eight subdomains were opened for the requests.
Before image slicing, the image in each layer was
transformed into a square. The longest side length was taken
as the side length to create a new square canvas. The upper 3.3. Rendering Efficiency. Currently, there are three main
left corner of the original image was overlapped with the online WebGIS rendering methods for processing GRIB,
upper left corner of the square, and the remainder of the NetCDF, and other file formats: GeoJSON processing, bi-
square was filled in white. Thus, layer 0 (7000 ∗ 7000), layer 1 nary compression, and grayscale compression. Table 2 lists
Complexity 7

lons,lats

sc
re
Screen distance of Visible area

en
D
in map

lone,late

or
gS
Z
Pyramid level
or
gS
Z/
2

Layer 0 Layer 1
Figure 4: The area relation between the visible screen range and the diagonal width of each layer.

Table 1: Slices after compression.

Compression to PNG Range of the single-slice file size Number of Distance between adjacent grid
Layer Resolution
(kB) (kB) slices points
0 7000 × 7000 1611.9 [0.6, 79.3] 256 0.01° (1 km)
1 3500 × 3500 543.0 [0.6, 57.3] 64 0.02° (2 km)
2 1750 × 1750 174.3 [0.9, 35.8] 16 0.04° (4 km)
3 875 × 875 56.4 [9.5, 20.4] 4 0.08° (8 km)
4 437 × 437 17.6 [17.6, 17.6] 1 0.16° (16 km)

Comparison of transmission timeliness

6000

5000
Transmission timeliness (ms)

4000

3000

2000

1000

0
0 10000 20000 30000 40000 50000 60000
File size (kB)
Whole compressed file
All single-slice files
Figure 5: Comparison of transmission timeliness between the whole compression file and the slices.
8 Complexity

Table 2: Comparison of the various treatment results and the the capabilities of the browser, reducing the pressure on the
rendering efficiency. server and taking advantage of combining cloud computing
Treatment Result (MB) Rendering time with sliding windows.
GeoJSON transformation 253 2.5 s
Binary compression 76 485 ms Data Availability
Grayscale image compression 89 287 ms
High-low 8-bit compression 1.3 145 ms The data were saved as Grib2 format, which can be downloaded
from http://image.data.cma.cn/test/Z_SURF_C_BABJ_P_CM
PA_RT_CHN_0P01_HOR-PRE-20210720.rar, and the data-
the rendering efficiencies of the above methods and the file can be decoded by the software of Panoply (https://www.
method in this article for a 120 MB file. The same WebGL giss.nasa.gov/tools/panoply/).
and WebGIS rendering technology was used for all pro-
cesses. The results indicate that the rendering efficiency of Conflicts of Interest
the high-low 8-bit compression method reduced the time
consumption to the order of milliseconds, thus making this The authors declare that there are no conflicts of interest
the optimal method. regarding the publication of this article.

4. Conclusion and Future Work Authors’ Contributions

Due to climate and weather phenomena such as global He-Ping Yang and Ying-Rui Sun conceptualized the study
climate change and the frequent occurrence of extreme and developed the original research program; Nan Chen and
weather, the demands for meteorological services have been Jing-Hua Chen analyzed the data; Qi Wang and Ming Yang
increasing in relation to various social activities and in- handled the data; He-Ping Yang and Xiao-Wei Jiang wrote
dustries. Accordingly, there is a need for methods of visu- the original draft; Zi-Mo Huo and Ming-Nong Feng com-
alizing the spatiotemporal characteristics of meteorological pared the results of the different methods. All authors have
big data for disaster prevention and mitigation for various read and agreed to the published version of the manuscript.
social activities and industries. Based on the Vue architec-
ture of HTML5, the scheme proposed in this article can be Acknowledgments
used to quickly visualize grid data of approximately 100 MB
in size with PC and mobile browsers as carriers. The mul- This work was supported in part by the National Key R&D
titerminal rendering of technical data is accomplished by Program of China (2016YFA0600301); a Major Project of
combining Python, Node.js, HTML5, Mapbox, and other Zhejiang Province, China (Grant 2021C02036); and the Key
technologies. The proposed process can support efficient Research and Development Project of China (Grant
WebGIS rendering of various kinds of large grid data files, 2018YFC1505601). The authors are thankful to Profs. Hui
thus providing a solution for quickly visualizing industrial Gao, Qingyun Zhang, and Xia Xu for helpful discussions.
data and spatial big data after fusion and improving the
efficiency of installation-free visualization of grid big data in References
browsers.
Data compression: Various compression algorithms, [1] M. B. G. Martı́n, “Weather, climate and tourism a geo-
including binary compression, grayscale map compression, graphical perspective,” Annals of Tourism Research, vol. 32,
and high-low 8-bit compression, were compared in terms of no. 3, pp. 571–591, 2005.
the compression ratio and the loss ratio. High-low 8-bit [2] L. Bright and D. Maier, Deriving and Managing Data Products
in an Environmental Observation and Forecasting System,” the
compression was selected because it enables the visual
2005 CIDR Conference, pp. 162–173, 2005.
display of meteorological values accumulated over periods [3] T. N. Palmer, J. Barkmeijer, R. Buizza, and T. Petroliagis, “The
such as years, months, and days. However, the use of this ECMWF ensemble prediction system,” Meteorological Ap-
compression algorithm is limited due to the large scale of the plications, vol. 4, no. 4, pp. 301–304, 1997.
accumulated values and the need for visualization accuracy [4] Y. Ota, J. C. Derber, E. Kalnay et al., “Ensemble-based ob-
up to three significant digits after the decimal point. servation impact estimates using the NCEP GFS,” Tellus A:
Slicing: Pyramid slicing met the efficiency requirements Dynamic Meteorology and Oceanography, vol. 65, no. 1,
of this study. The visualization efficiency could be further p. 20038, 2013.
improved by adopting other algorithms, such as the quad- [5] R. Zhang and X. Shen, “On the development of the
tree algorithm, based on specific requirements. GRAPES――A new generation of the national operational
Transmission and rendering: In this study, the image NWP system in China,” Science Bulletin, vol. 53, no. 22,
pp. 3429–3432, 2008.
rendering data were separated from the original data. The
[6] J He, K Yang, W Tang et al., “The first high-resolution me-
data of layer 0 were original, while the data in the other layers teorological forcing dataset for land process studies over
were rendered on the web page, thus preserving the ei- China,” Scientific Data, vol. 7, no. 1, pp. 25–11, 2020.
genvalues in the visualization, which is preferable to grid [7] S. E. Haupt and B. Kosović, “Variable generation power
pumping and visualization. Furthermore, the rendering forecasting as a big data problem,” IEEE Transactions on
method based on the WebGL technical framework fully uses Sustainable Energy, vol. 8, no. 2, pp. 725–732, 2016.
Complexity 9

[8] C. S. Zender, “Analysis of self-describing gridded geoscience [26] V. Lytvyn, O. Pashchetnyk, O. Klymovych et al., “Assessment
data with netCDF Operators (NCO),” Environmental Mod- of the hydro-meteorological conditions impact on the combat
elling & Software, vol. 23, no. 10-11, pp. 1338–1342, 2008. troops operations preparation and conduct in the geo-in-
[9] P. Berrisford, R. Brugge, L. Steenman-Clark, and D. Li, “The formation subsystem of the automated battlefield manage-
UGAMP use and diagnosis of the ECMWF meteorological ment system,” CEUR Workshop Proceedings, vol. 1,
analyses,” Systems Analysis Modelling Simulation, vol. 42, pp. 1063–1076, 2021.
no. 11, pp. 1615–1621, 2002. [27] R. S. Prastika, A. N. Afandi, and D. Prihanto, “Mitigation of
[10] M. Ninyerola, X. Pons, and J. M. Roure, “A methodological the alternative energy for the wind farm center considering
approach of climatological modelling of air temperature and temperature and wind speed,” MATEC Web of Conferences,
precipitation through GIS techniques,” International Journal vol. 204, p. 04013, 2018.
of Climatology, vol. 20, no. 14, pp. 1823–1841, 2000. [28] Y. Jiang, S. Han, C. Shi, T. Gao, H. Zhen, and X. Liu,
[11] R. B. Schmunk, Panoply netCDF, HDF and GRIB Data Viewer, “Evaluation of HRCLDAS and ERA5 datasets for near-surface
NASA Goddard Institute for Space Studies, 2017. wind over hainan island and south China sea,” Atmosphere,
[12] Y. Q. Wang, “MeteoInfo: GIS software for meteorological data vol. 12, no. 6, p. 766, 2021.
visualization and analysis,” Meteorological Applications, [29] B. Kastanakis, Mapbox Cookbook, Packt Publishing Ltd, 2016.
vol. 21, no. 2, pp. 360–368, 2014. [30] G. Qiu and J. Chen, Web-based 3D Map Visualization Using
[13] H Wu, W Zheng, B Luo et al., “Decision making meteoro- WebGL,” 2018 13th IEEE Conference On Industrial Electronics
logical services system based on geographic information And Applications (ICIEA), pp. 759–763, 2018.
[31] T. Parisi, WebGL: Up and Running, O’Reilly Media, Inc., 2012.
system,” International Joint Conference on Computational
[32] M. Samir, M. Azab, M. R. M. Rizk, and N. Sadek, “PYGRID: a
Sciences and Optimization, vol. 2, pp. 53–55, 2009.
software development and assessment framework for grid-
[14] F. Berman, A. Chien, K. Cooper et al., “The GrADS project:
aware software defined networking,” International Journal of
software support for high-level grid application develop-
Network Management, vol. 28, no. 5, p. e2033, 2018.
ment,” International Journal of High Performance Computing
[33] S. Gao and V. Gruev, “Bilinear and bicubic interpolation
Applications, vol. 15, no. 4, pp. 327–344, 2001. methods for division of focal plane polarimeters,” Optics
[15] D. Murray, J. McWhirter, Y. Ho et al., “The IDV at 5: new Express, vol. 19, no. 27, pp. 26161–26173, 2011.
features and future plans,” Proceedings of 25th Conference on [34] N. Nikiforakis, M. Balduzzi, L. Desmet, F. Piessens, and
International Interactive Information and Processing Systems W. Joosen, “Soundsquatting: uncovering the use of homo-
(IIPS) for Meteorology, Oceanography, and Hydrology, vol. 7, phones in domain squatting,” Lecture Notes in Computer
no. 2, 2009. Science, vol. 8783, pp. 291–308, 2014.
[16] R. Nogueira and E. M. Cutrim, “Applications of “integrated [35] W. Busch, B. T. Moore, B. Martsberger et al., “A microfluidic
data viewer” (IDV) in the classroom,” Advances in Geo- device and computational platform for high-throughput live
sciences, vol. 8, pp. 63–67, 2006. imaging of gene expression,” Nature Methods, vol. 9, no. 11,
[17] J. Gaigalas, L Di, and Z. Sun, “Advanced cyberinfrastructure pp. 1101–1106, 2012.
to enable search of big climate datasets in THREDDS,” ISPRS [36] P. Halliday, Vue. Js 2 Design Patterns and Best Practices: Build
International Journal of Geo-Information, vol. 8, no. 11, p. 494, enterprise-ready, Modular Vue. Js Applications with Vuex and
2019. Nuxt, Packt Publishing Ltd, 2018.
[18] M. Linaje, J. C. Preciado, and F. Sanchez-Figueroa, “Engi- [37] Y. Wang, K. Dai, Z. Zong et al., “Quantitative precipitation
neering rich internet application user interfaces over legacy forecasting using multi-model blending with supplemental
web models,” IEEE internet computing, vol. 11, no. 6, grid points: experiments and prospects in China,” Journal of
pp. 53–59, 2007. Meteorological Research, vol. 35, no. 3, pp. 521–536, 2021.
[19] S. Popelka, A. Vondrakova, and P. Hujnakova, “Eye-tracking
evaluation of weather web maps,” ISPRS International Journal
of Geo-Information, vol. 8, no. 6, p. 256, 2019.
[20] S. Popić, D. Pezer, B. Mrazovac et al., “Performance evalu-
ation of using protocol buffers in the internet of things
communication,” in International Conference on Smart Sys-
tems and Technologies (SST), pp. 261–265, 2016.
[21] X. Shang, A Study on Efficient Vector Mapping with Vector
Tiles Based on Cloud Server Architecture, Graduate Studies,
2015.
[22] N. Yap, M. Gong, R. K. Naha, and A. Mahanti, “Machine
learning-based modelling for museum visitations prediction,”
in Proceedings of the 2020 International Symposium on Net-
works, Computers and Communications (ISNCC), pp. 1–7,
Shenzhen, China, 2020.
[23] L. S. Hsu and O. Obe, PostGIS in Action, Manning Publi-
cations Co., 2015.
[24] B. N. Hill, An Analysis of the Factors that Influence the
Sargassum Migratory Loop, Diss., 2016.
[25] K. S. Yarygin, B. A. Kovarsky, T. S. Bibikova, D. S. Melnikov,
A. V. Tyakht, and D. G. Alexeev, “ResistoMap-online visu-
alization of human gut microbiota antibiotic resistome,”
Bioinformatics, vol. 33, no. 14, pp. 2205-2206, 2017.