REST API Best Practices for Company Data Lookups

Calling a company data API in development is easy. Making it reliable in production is where the real work begins. Network failures, rate limits, upstream outages, and stale caches can all degrade your user experience if you do not handle them properly.

This article covers production-tested patterns for integrating company data APIs like KVKBase, the Dutch KVK API, or the EU VIES service into your application.

1. Smart Caching

Company data changes infrequently. A company’s name, address, and KVK number are stable for months or years. This makes aggressive caching both safe and valuable.

Recommended TTL values

Data type	TTL	Rationale
Active company profile	24 hours	Addresses or names rarely change overnight
Inactive/dissolved company	7 days	Status will not change back
Search results	1 hour	New companies register daily
Failed lookups (404)	1 hour	Company may have just registered
VIES validation (valid)	7 days	VAT status rarely changes
VIES validation (invalid)	1 hour	May be temporary VIES downtime

Implementation

class CompanyDataCache {
  constructor(store) {
    this.store = store; // Redis, Memcached, or in-memory
  }

  async get(key) {
    const entry = await this.store.get(key);
    if (!entry) return null;

    const parsed = JSON.parse(entry);
    if (Date.now() > parsed.expiresAt) {
      await this.store.delete(key);
      return null;
    }

    return parsed.data;
  }

  async set(key, data, ttlSeconds) {
    const entry = {
      data,
      expiresAt: Date.now() + (ttlSeconds * 1000),
      cachedAt: Date.now()
    };

    await this.store.set(key, JSON.stringify(entry), ttlSeconds);
  }

  ttlForLookup(result) {
    if (!result) return 3600;                // Not found: 1 hour
    if (!result.isActive) return 604800;     // Inactive: 7 days
    return 86400;                            // Active: 24 hours
  }
}

Cache key design

Use structured, predictable cache keys:

function cacheKey(operation, ...parts) {
  return `company:${operation}:${parts.join(':')}`;
}

// Examples:
// company:lookup:12345678
// company:search:bol.com:limit=10
// company:vies:NL:123456789B01

2. Retry with Exponential Backoff

Transient failures happen. A retry strategy with exponential backoff handles them gracefully without overwhelming the upstream service.

async function withRetry(fn, options = {}) {
  const {
    maxRetries = 3,
    baseDelay = 1000,
    maxDelay = 30000,
    retryableErrors = [502, 503, 504, 429]
  } = options;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      const isRetryable = error.status
        ? retryableErrors.includes(error.status)
        : error.code === 'ECONNRESET' || error.code === 'ETIMEDOUT';

      if (!isRetryable || attempt === maxRetries) {
        throw error;
      }

      const delay = Math.min(
        baseDelay * Math.pow(2, attempt) + Math.random() * 1000,
        maxDelay
      );

      // If rate limited, use Retry-After header
      if (error.status === 429 && error.retryAfter) {
        await sleep(error.retryAfter * 1000);
      } else {
        await sleep(delay);
      }
    }
  }
}

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

Usage

const result = await withRetry(
  () => kvkbase.lookup('12345678'),
  { maxRetries: 3, baseDelay: 1000 }
);

3. Rate Limit Handling

Most company data APIs enforce rate limits. Handling them properly means your application degrades gracefully instead of crashing.

Reading rate limit headers

class RateLimitAwareClient {
  constructor(baseUrl, apiKey) {
    this.baseUrl = baseUrl;
    this.apiKey = apiKey;
    this.remaining = Infinity;
    this.resetAt = 0;
  }

  async request(path) {
    // Proactively wait if we know we are out of quota
    if (this.remaining <= 0 && Date.now() < this.resetAt) {
      const waitTime = this.resetAt - Date.now();
      await sleep(waitTime);
    }

    const response = await fetch(`${this.baseUrl}${path}`, {
      headers: { 'Authorization': `Bearer ${this.apiKey}` }
    });

    // Update rate limit state from response headers
    this.remaining = parseInt(response.headers.get('X-RateLimit-Remaining') || 'Infinity');
    this.resetAt = parseInt(response.headers.get('X-RateLimit-Reset') || '0') * 1000;

    if (response.status === 429) {
      const retryAfter = parseInt(response.headers.get('Retry-After') || '60');
      throw { status: 429, retryAfter, message: 'Rate limited' };
    }

    if (!response.ok) {
      throw { status: response.status, message: response.statusText };
    }

    return response.json();
  }
}

Request queuing for bulk operations

When processing many lookups, use a queue to stay within rate limits:

class RequestQueue {
  constructor(requestsPerSecond = 5) {
    this.interval = 1000 / requestsPerSecond;
    this.lastRequest = 0;
    this.queue = [];
    this.processing = false;
  }

  async enqueue(fn) {
    return new Promise((resolve, reject) => {
      this.queue.push({ fn, resolve, reject });
      this.process();
    });
  }

  async process() {
    if (this.processing) return;
    this.processing = true;

    while (this.queue.length > 0) {
      const elapsed = Date.now() - this.lastRequest;
      if (elapsed < this.interval) {
        await sleep(this.interval - elapsed);
      }

      const { fn, resolve, reject } = this.queue.shift();
      this.lastRequest = Date.now();

      try {
        resolve(await fn());
      } catch (error) {
        reject(error);
      }
    }

    this.processing = false;
  }
}

4. Circuit Breaker for External Services

The VIES VAT validation service is known for intermittent outages. A circuit breaker prevents your application from wasting time on requests that will fail.

class CircuitBreaker {
  constructor(options = {}) {
    this.failureThreshold = options.failureThreshold || 5;
    this.resetTimeout = options.resetTimeout || 60000;
    this.failures = 0;
    this.state = 'CLOSED'; // CLOSED, OPEN, HALF_OPEN
    this.lastFailure = null;
  }

  async execute(fn) {
    if (this.state === 'OPEN') {
      if (Date.now() - this.lastFailure > this.resetTimeout) {
        this.state = 'HALF_OPEN';
      } else {
        throw new Error('Circuit breaker is OPEN');
      }
    }

    try {
      const result = await fn();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  onSuccess() {
    this.failures = 0;
    this.state = 'CLOSED';
  }

  onFailure() {
    this.failures++;
    this.lastFailure = Date.now();
    if (this.failures >= this.failureThreshold) {
      this.state = 'OPEN';
    }
  }
}

// Usage with VIES
const viesBreaker = new CircuitBreaker({
  failureThreshold: 3,
  resetTimeout: 120000 // 2 minutes
});

async function validateVat(countryCode, vatNumber) {
  try {
    return await viesBreaker.execute(() =>
      viesClient.validate(countryCode, vatNumber)
    );
  } catch (error) {
    if (error.message === 'Circuit breaker is OPEN') {
      return { valid: null, message: 'VAT validation temporarily unavailable' };
    }
    throw error;
  }
}

5. Response Normalization

Different company data APIs return data in different formats. Normalize responses into a consistent internal format:

function normalizeCompanyData(source, rawData) {
  switch (source) {
    case 'kvkbase':
      return {
        kvkNumber: rawData.kvkNumber,
        name: rawData.tradeName,
        legalForm: rawData.legalForm,
        address: {
          street: rawData.address?.street,
          houseNumber: rawData.address?.houseNumber,
          postalCode: rawData.address?.postalCode,
          city: rawData.address?.city,
          country: 'NL'
        },
        vatNumber: rawData.vatNumber,
        isActive: rawData.isActive,
        source: 'kvkbase',
        retrievedAt: new Date().toISOString()
      };

    case 'kvk-official':
      return {
        kvkNumber: rawData.kvkNummer,
        name: rawData.naam,
        legalForm: rawData.rechtsvorm,
        address: {
          street: rawData.straatnaam,
          houseNumber: rawData.huisnummer,
          postalCode: rawData.postcode,
          city: rawData.plaats,
          country: 'NL'
        },
        vatNumber: null, // Official KVK API does not include VAT
        isActive: rawData.actief === 'Ja',
        source: 'kvk-official',
        retrievedAt: new Date().toISOString()
      };

    default:
      throw new Error(`Unknown source: ${source}`);
  }
}

6. Timeouts and Fallbacks

Set explicit timeouts and define fallback behavior:

async function lookupWithFallback(kvkNumber) {
  // Try primary source with timeout
  try {
    const controller = new AbortController();
    const timeout = setTimeout(() => controller.abort(), 5000);

    const result = await fetch(
      `https://api.kvkbase.nl/api/v1/lookup/${kvkNumber}`,
      {
        headers: { 'Authorization': `Bearer ${API_KEY}` },
        signal: controller.signal
      }
    );

    clearTimeout(timeout);
    return await result.json();
  } catch (error) {
    // Fall back to cache, even if stale
    const staleCache = await cache.getStale(kvkNumber);
    if (staleCache) {
      return { ...staleCache, _stale: true };
    }

    throw error;
  }
}

Putting It All Together

A production-grade company data client combines all these patterns:

class CompanyDataService {
  constructor(config) {
    this.client = new RateLimitAwareClient(config.baseUrl, config.apiKey);
    this.cache = new CompanyDataCache(config.cacheStore);
    this.queue = new RequestQueue(config.requestsPerSecond);
    this.viesBreaker = new CircuitBreaker({ failureThreshold: 3 });
  }

  async lookup(kvkNumber) {
    const cached = await this.cache.get(`lookup:${kvkNumber}`);
    if (cached) return cached;

    const result = await this.queue.enqueue(() =>
      withRetry(() => this.client.request(`/lookup/${kvkNumber}`))
    );

    const ttl = this.cache.ttlForLookup(result);
    await this.cache.set(`lookup:${kvkNumber}`, result, ttl);

    return result;
  }
}

These patterns are not specific to KVK data — they apply to any external API integration. But company data lookups, with their mix of stable data and unreliable upstream services, are a particularly good fit for this level of resilience engineering.