Java serialization filters: constrain classes and object graphs

This complete program targets Java 11. Its displayed output is checked by the tutorial validation script.

Limit the specific legacy contract

A legacy migration file is expected to contain one small primitive int array. The fixture allows only int[] and caps its length, graph depth, references and reported bytes. It also caps the input byte array before opening the stream. A larger array and a different primitive array class are rejected independently.

This deliberately narrow filter has no application-defined serializable classes and no callbacks. Accepting an entire package would enlarge the trusted code surface. Prefer an explicit data format for new boundaries; a filter is a constraint on an existing serialization path, not a reason to accept arbitrary object streams.

Validate the returned value too

A filter receives graph information and sometimes no class. Return UNDECIDED for a metric-only callback after checking bounds, and reject unknown non-null classes. After reading, check the root type and application values. A class allowlist does not prove that an accepted array contains a valid business record.

The fixture checks complete locally generated files, not hostile network fuzzing. Stream metrics are not a substitute for a transport byte limit: filtering callbacks are not guaranteed to occur at every byte. Bounded byte input should be designed before the object stream is constructed.

Working program

import java.io.*;
public class LegacyBatchFilter {
    static byte[] encode(Object value) throws IOException {
        ByteArrayOutputStream bytes = new ByteArrayOutputStream();
        try (ObjectOutputStream out = new ObjectOutputStream(bytes)) { out.writeObject(value); }
        return bytes.toByteArray();
    }
    static int[] read(byte[] bytes) throws Exception {
        if (bytes.length > 1024) throw new IOException("byte limit");
        try (ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(bytes))) {
            in.setObjectInputFilter(info -> {
                if (info.depth() > 2 || info.references() > 8 || info.streamBytes() > 1024 || info.arrayLength() > 4)
                    return ObjectInputFilter.Status.REJECTED;
                Class<?> type = info.serialClass();
                return type == null ? ObjectInputFilter.Status.UNDECIDED
                    : type == int[].class ? ObjectInputFilter.Status.ALLOWED : ObjectInputFilter.Status.REJECTED;
            });
            Object value = in.readObject();
            if (!(value instanceof int[])) throw new IOException("root type");
            return (int[]) value;
        }
    }
    public static void main(String[] args) throws Exception {
        System.out.println(read(encode(new int[]{12, 18})).length);
        try { read(encode(new int[5])); }
        catch (InvalidClassException rejected) { System.out.println("length rejected"); }
        try { read(encode(new long[]{12})); }
        catch (InvalidClassException rejected) { System.out.println("class rejected"); }
    }
}

Output

2
length rejected
class rejected

Costs and boundaries

Serialization walks the accepted object graph and allocates reconstructed values. The fixture bounds the byte array and array length; it does not establish a universal CPU limit or sandbox code in accepted classes. Do not deserialize untrusted objects merely because a filter exists.

Common Mistakes

• Avoid broad package allowlists for a narrow migration format.
• Cap transport input independently of filter callbacks.
• Validate the root and its values after reading.

Read next

Java byte streams: partial reads and bounded copying, Java try-with-resources: close order and suppressed failures, Spring MVC request validation: reject invalid commands before mutation.