Python Serialize NumPy ndarray into JSON

You are here because when you try to encode/serialize NumPy array into a JSON format, you received a TypeError: Object of type ndarray is not JSON serializable. In this article, I will show you how to make NumPy array JSON serializable so that you can convert any NumPy array into JSON formatted data.

As you know The built-in json module of Python can only handle primitives types that have a direct JSON equivalent (e.g., dictionary, lists, strings, Numbers, None, etc.). To serialize the multidimensional array into JSON, We need to write a custom JSON Encoder.

Custom JSON Encoder to Serialize NumPy ndarray

Python json module has a JSONEncoder class, we can extend it to get more customized output. i.e., you will have to subclass JSONEncoder so you can implement custom NumPy JSON serialization.

When we extend the JSONEncoder class, we will extend its JSON encoding scope by overriding the default() method which will be used when we execute JSONEncoder.encode(numpyArray).

Use the cls kwarg of the json.dump() and json.dumps() method to call our custom JSON Encoder, which will convert NumPy array into JSON formatted data.

Example: json.dumps(cls=NumPyArrayEncoder)

To serialize Numpy array into JSON we need to convert it into a list structure using a tolist() function. Let’s see the demo.

Encode and Decode NumPy array to and from JSON

In this example, we try to serialize the NumPy Array into JSON String.

import json
from json import JSONEncoder
import numpy

class NumpyArrayEncoder(JSONEncoder):
    def default(self, obj):
        if isinstance(obj, numpy.ndarray):
            return obj.tolist()
        return JSONEncoder.default(self, obj)

numpyArrayOne = numpy.array([[11, 22, 33], [44, 55, 66], [77, 88, 99]])

# Serialization
numpyData = {"array": numpyArrayOne}
encodedNumpyData = json.dumps(numpyData, cls=NumpyArrayEncoder)  # use dump() to write array into file
print("Printing JSON serialized NumPy array")
print(encodedNumpyData)

# Deserialization
print("Decode JSON serialized NumPy array")
decodedArrays = json.loads(encodedNumpyData)

finalNumpyArray = numpy.asarray(decodedArrays["array"])
print("NumPy Array")
print(finalNumpyArray)

Output:

Printing JSON serialized NumPy array
{"array": [[11, 22, 33], [44, 55, 66], [77, 88, 99]]}

Decode JSON serialized NumPy array
NumPy Array
[[11 22 33]
 [44 55 66]
 [77 88 99]]

Note: We used numpy.asarray() to convert data into NumPy array

Encode NumPy array into JSON and write it in a file

In most scenarios, we need to store JSON serialized NumPy array into a file so we can use it in different systems.

In this example, we will do the following:

• Convert two NumPy arrays into JSON and write it into a JSON file
• Read the JSON file, which contains JSON serialized NumPy array and convert it into actual NumPy Array.

import numpy
from json import JSONEncoder
import json

class NumpyArrayEncoder(JSONEncoder):
    def default(self, obj):
        if isinstance(obj, numpy.ndarray):
            return obj.tolist()
        return JSONEncoder.default(self, obj)

numpyArrayOne = numpy.array([[11 ,22, 33], [44, 55, 66], [77, 88, 99]])
numpyArrayTwo = numpy.array([[51, 61, 91], [121 ,118, 127]])

# Serialization
numpyData = {"arrayOne": numpyArrayOne, "arrayTwo": numpyArrayTwo}
print("serialize NumPy array into JSON and write into a file")
with open("numpyData.json", "w") as write_file:
    json.dump(numpyData, write_file, cls=NumpyArrayEncoder)
print("Done writing serialized NumPy array into file")

# Deserialization
print("Started Reading JSON file")
with open("numpyData.json", "r") as read_file:
    print("Converting JSON encoded data into Numpy array")
    decodedArray = json.load(read_file)

    finalNumpyArrayOne = numpy.asarray(decodedArray["arrayOne"])
    print("NumPy Array One")
    print(finalNumpyArrayOne)
    finalNumpyArrayTwo = numpy.asarray(decodedArray["arrayTwo"])
    print("NumPy Array Two")
    print(finalNumpyArrayTwo)

Output:

serialize NumPy array into JSON and write into a file
Done writing serialized NumPy array into file
Started Reading JSON file
Converting JSON encoded data into Numpy array
NumPy Array One
[[11 22 33]
 [44 55 66]
 [77 88 99]]
NumPy Array Two
[[ 51  61  91]
 [121 118 127]]

Encode all NumPy types correctly into JSON

import json
from json import JSONEncoder
import numpy as np

class NumpyArrayEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.integer):
            return int(obj)
        elif isinstance(obj, np.floating):
            return float(obj)
        elif isinstance(obj, np.ndarray):
            return obj.tolist()
        else:
            return super(NumpyArrayEncoder, self).default(obj)

# Serialization
numPyData = {"id": 25, "floatSample": np.float32(1.2), "intSample":np.int32(42), "arangeSample": np.arange(12)}
encodedNumpyData = json.dumps(numPyData, cls=NumpyArrayEncoder)  # use dump() to write array into file
print("Printing JSON serialized NumPy array")
print(encodedNumpyData)

# Deserialization
print("Decode JSON serialized NumPy array")
decodedArrays = json.loads(encodedNumpyData)

numPyFloat = np.asarray(decodedArrays["floatSample"])
print("numPy Float")
print(numPyFloat)

numPyInt = np.asarray(decodedArrays["intSample"])
print("numPy Integer")
print(numPyInt)

numPyArange = np.asarray(decodedArrays["arangeSample"])
print("numPy arange")
print(numPyArange)

Output:

Printing JSON serialized NumPy array
{"id": 25, "floatSample": 1.2000000476837158, "intSample": 42, "arangeSample": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]}
Decode JSON serialized NumPy array
numPy Float
1.2000000476837158
numPy Integer
42
numPy arange
[ 0  1  2  3  4  5  6  7  8  9 10 11]

Use panads to_json method to serialize NumPy ndarray into JSON

If you use pandas for data manipulation, you can use to_json function on Series. The pandas.Series.to_json convert the object to a JSON string. we can pass NumPy array to it to get the JSON representation.

import pandas as pd
import numpy

numpyArray = numpy.array([[51, 61, 91], [121, 118, 127]])

df = pd.DataFrame(numpyArray, index=['row 1', 'row 2'], columns=['col 1', 'col 2', 'col 3'])
df = df.to_json(orient='index')

print("Json Encoded Array")
print(df)

Output:

Json Encoded Array
{"row 1":{"col 1":51,"col 2":61,"col 3":91},"row 2":{"col 1":121,"col 2":118,"col 3":127}}

So What Do You Think?

I want to hear from you. What do you think of this article? Or maybe I missed one of the ways to serialize NumPy ndarray into JSON. Either way, let me know by leaving a comment below.

Also, try to solve the Python JSON Exercise to have a better understanding of Working with JSON Data in Python.