8000 fix: 일본 상품 상세를 무료 HTTP 중계로 복구 by hmmhmmhm · Pull Request #292 · hmmhmmhm/daiso-mcp · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,6 @@ OY_ACCESS_CLIENT_SECRET=
# CGV_RELAY_TOKEN=your_dtryx_relay_token
# CGV_ACCESS_CLIENT_ID=your_access_client_id
# CGV_ACCESS_CLIENT_SECRET=your_access_client_secret

# 일본 상품 상세만 기존 DTRYX 인증 HTTP 중계로 조회할 때 true
DAISO_JP_PRODUCT_RELAY_ENABLED=false
11 changes: 11 additions & 0 deletions docs/daiso-jp-network-analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,14 @@ npx daiso jp-stores 東京都渋谷区 --json
정기 deep/full의 CLI 계약 검사에 일본 상세 `/api/daiso-jp/products/4550746075917`을 추가했습니다. 일본 REST 상품·상세·매장에도 다른 서비스와 같은 인증된 `x-health-check-force-fresh: true` 캐시 우회를 적용했습니다. 인증 없는 요청은 기존 캐시 정책을 유지하며, 검사 실패를 성공 캐시로 숨기지 않습니다.

같은 날 운영 반영 후 인증된 캐시 우회 상세503 및 CLI/MCP 실패가 다시 재현됐고, 별도 공식 코드 검색은 정확한 상품으로200을 반환했습니다. 후속 수정에서는 상품 코드 검색을 먼저 확보한 뒤 상세 HTML로 온라인 판매 여부를 보강합니다. 상세403이면 이미 확보한 정확한ID만 제한 안내와 함께 반환하며, 검색이 실패해도 상세 성공은 사용합니다. 상세404·429·500이나 파싱 오류는 검색 성공으로 감추지 않습니다. 조회는 검색1회·상세1회로 제한하며 재시도를 추가하지 않습니다. 직접 상세가 성공하던 환경에는 검색1회가 추가되는 비용이 있습니다. 이 순서 변경의 실제 운영 검증과 원본403의 영구 해소 여부는 구분합니다.


### 최종 상세 전송: 명시적 Mac HTTP 중계

앞의 검색 선조회 순서 변경은 실제 운영403을 해소하지 못해 철회했습니다. 로컬과 자체 배포에서 별도 설정이 없으면 기존 상세→403시 정확한코드검색 흐름을 유지합니다. 운영 Worker는 `DAISO_JP_PRODUCT_RELAY_ENABLED="true"`를 명시하고 기존 DTRYX_RELAY_URL/TOKEN/ACCESS_CLIENT_ID/ACCESS_CLIENT_SECRET 설정으로 일본 상세만 HTTP 중계에 연결합니다. 이 플래그가 없으면 DTRYX 설정만으로 일본 전송이 바뀌지 않습니다.

`POST /v1/daiso-jp/product`는 `{ "id": "4550746075917" }`만 받으며 임의 URL을 허용하지 않습니다. Mac에서 공식 `https://jp.daisonet.com/products/{id}.js`만 읽고 요청ID와 대조합니다. 리다이렉트는 거절하고 본문2MiB·전체15초 제한을 적용합니다. Worker도 같은 파서로 응답ID를 다시 검증합니다. 재시작할 때 공유 파서의 `src/services/daisoJp/parser.ts`와 types.ts도 runtime에 배치해야 합니다.

상품 검색·매장 검색은 기존 직접 경로를 유지하며 상세에는 브라우저·가상기기·새 로그인·유료 프록시가 필요하지 않습니다. 상세는 연결시험 후 재요청하는 대신 명시적 중계 경로를 사용합니다. 공통 DTRYX 중계의 건강 상태는 기존5분 Cron으로 확인되며 일본 상세는 deep/full CLI 계약에서 실제 조회합니다. 별도 직접상세 probe를 늘리지 않습니다.

일본 상세는 DTRYX/CGV와 동일한 호출 예산 원장을 공유하지만 서비스별 접수·동시성 큐는 별도입니다. 상품정보 캐시는 기존 REST와 동일하게30분이며 인증된 force-fresh는 Worker와 중계 양쪽을 우회합니다. 자체 할당량429·중계인증·원본차단·미확인판매 여부를 성공이나 재고0으로 바꾸지 않습니다. 원본 HTTP상태는 비공개 중계로그에서 확인하고 Worker 공개진단의 upstreamStatus는 중계응답 상태입니다. 일본 오프라인 매장 재고 미지원 범위는 그대로입니다.
22 changes: 22 additions & 0 deletions scripts/relay/daiso-jp.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/** 고정 일본 다이소 상품 JSON만 제공하며 임의 URL을 받지 않습니다. */
import { parseProductDetail } from '../../src/services/daisoJp/parser.js';
import { createHttpRelay, RelayError, abortable, readJson, type HttpRelayOptions } from './http-relay.js';
export function createJapanProductRelay(token: string, options: Pick<HttpRelayOptions, 'takeQuota' | 'fetcher' | 'onEvent'>) {
return createHttpRelay(token, {
...options, prefix: '/v1/daiso-jp/', operations: ['product'], cacheTtl: () => 1800000,
validate(value) {
if (!value || typeof value !== 'object' || Array.isArray(value)) throw new RelayError(400);
const p = value as Record<string, unknown>;
if (Object.keys(p).length !== 1 || typeof p.id !== 'string' || !/^\d{13}$/.test(p.id)) throw new RelayError(400);
return { id: p.id };
},
async upstream(_operation, value, signal, fetcher) {
const { id } = value as { id: string };
const response = await abortable(fetcher(new URL(`https://jp.daisonet.com/products/${id}.js`), { redirect: 'manual', signal }), signal);
if (response.status !== 200) { void response.body?.cancel().catch(() => undefined); throw new RelayError(502, undefined, undefined, response.status); }
const data = await readJson(response.body, 2 * 1024 * 1024, signal, 502);
parseProductDetail(data, id);
return data;
},
});
}
4 changes: 4 additions & 0 deletions scripts/relay/dtryx-start.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { pathToFileURL } from 'node:url';
import { createFileQuota, assertSeparateQuotaDirectories, CONVENIENCE_QUOTA_LIMITS } from './quota.js';
import { createConvenienceRelay } from './convenience.js';
import { createGs25SessionTransport } from './gs25-session.js';
import { createJapanProductRelay } from './daiso-jp.js';
import { createCgvRelay } from './cgv.js';
import { createDtryxRelay } from './dtryx.js';

Expand Down Expand Up @@ -72,8 +73,11 @@ async function main() {
}
return response;
}, log.append);
const japanRelay = createJapanProductRelay(token, { takeQuota, onEvent: log.append });
const japanHandler = observeRelay(japanRelay, log.append);
const handler = (request: Request) => new URL(request.url).pathname.startsWith('/v1/convenience/')
? convenience ? convenience(request) : Promise.resolve(Response.json({ error: 'Not found' }, { status: 404 }))
: new URL(request.url).pathname.startsWith('/v1/daiso-jp/') ? japanHandler(request)
: new URL(request.url).pathname.startsWith('/v1/cgv/') ? cgvHandler(request) : dtryxHandler(request);
const server = createServer(async (req, res) => {
const controller = new AbortController();
Expand Down
1 change: 1 addition & 0 deletions src/api/response.ts
8000
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import type { Context } from 'hono';
import { ServiceError, toServiceErrorDiagnostics, toStandardErrorDiagnostics } from '../core/errors.js';

export interface AppBindings {
DAISO_JP_PRODUCT_RELAY_ENABLED?: string;
DAILY_RATE_LIMITER?: DurableObjectNamespace;
UPSTREAM_ROUTE_HEALTH?: DurableObjectNamespace;
GEOCODE_GATEWAY?: DurableObjectNamespace;
Expand Down
6 changes: 4 additions & 2 deletions src/api/routes/daisoJpRoutes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { successResponse, errorResponse, serviceErrorResponse } from '../respons
import { withEdgeCache } from '../../utils/cache.js';
import { ServiceError } from '../../core/errors.js';
import { searchProducts, getProduct, findStores } from '../../services/daisoJp/client.js';
import { japanProductTransportFromBindings } from '../../services/daisoJp/productRelay.js';
import { jpQuery, jpPage, jpProductId } from '../../services/daisoJp/index.js';
import { isFreshOperationalRequest } from '../operationalFreshness.js';

Expand Down Expand Up @@ -34,8 +35,9 @@ export function registerDaisoJpRoutes(app: Hono<{ Bindings: AppBindings }>): voi
'INVALID_INPUT',
'상품 ID는 일본 다이소 검색 결과의 13자리 코드를 사용하세요.',
);
return withEdgeCache(c.req.url, { ttlSeconds: 1800, keyPrefix: 'daiso-jp-product-v1', bypassCache: await isFreshOperationalRequest(c) }, () =>
execute(c, () => getProduct(id.data)),
const bypassCache = await isFreshOperationalRequest(c);
return withEdgeCache(c.req.url, { ttlSeconds: 1800, keyPrefix: 'daiso-jp-product-v1', bypassCache }, () =>
execute(c, () => getProduct(id.data, japanProductTransportFromBindings(c.env, bypassCache))),
);
});
app.get('/api/daiso-jp/products', async (c) => {
Expand Down
4 changes: 2 additions & 2 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import { createDaisoService } from './services/daiso/index.js';
import { createOliveyoungService } from './services/oliveyoung/index.js';
import { createDtryxService } from './services/dtryx/index.js';
import { createMegaboxService } from './services/megabox/index.js';
import { japanProductTransportFromBindings } from './services/daisoJp/productRelay.js';
import { cgvTransportFromBindings } from './services/cgv/relayTransport.js';
import { createCgvService } from './services/cgv/index.js';
import { createLotteCinemaService } from './services/lottecinema/index.js';
Expand Down Expand Up @@ -58,7 +59,6 @@ export { DailyRateLimiter } from './durableObjects/dailyRateLimiter.js';
const 8000 SERVER_NAME = 'multi-service-mcp';
const SERVER_VERSION = '1.0.0';
const SESSION_HEADER = 'mcp-session-id';

// 세션별 MCP 전송 계층/서버를 유지해 GET/POST/DELETE를 일관 처리합니다.
const mcpSessions = new Map<
string,
Expand All @@ -76,7 +76,7 @@ const createRegistry = (bindings?: AppBindings) => {

registry.registerAll([
createDaisoService,
createDaisoJpService,
() => createDaisoJpService(japanProductTransportFromBindings(bindings)),
() =>
createGs25Service({ ...convenienceTransportFromBindings(bindings),
...geocodeBindings(bindings),
Expand Down
17 changes: 6 additions & 11 deletions src/services/daisoJp/client.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { ServiceError } from '../../core/errors.js';
import { fetchText, HttpError } from '../../utils/http.js';
import { parseProductPage, parseProductSearch, parseStoreSearch } from './parser.js';
import { requestJapanProductRelay, type JapanTransportOptions } from './productRelay.js';
import type { JapanProduct, JapanProductSearchResult, JapanStoreSearchResult } from './types.js';

const OPTIONS = { timeout: 20000, retries: 0 };
Expand Down Expand Up @@ -44,28 +45,22 @@ export async function searchProducts(query: string, page = 1): Promise<JapanProd
}));
}

export async function getProduct(id: string): Promise<JapanProduct> {
export async function getProduct(id: string, options: JapanTransportOptions = {}): Promise<JapanProduct> {
if (typeof id !== 'string' || !/^\d{13}$/.test(id)) {
throw new Error('상품 ID는 13자리 숫자여야 합니다.');
}
if (options.productRelay) return requestJapanProductRelay(id, options.productRelay);
return upstream(async () => {
// 차단 응답 뒤 검색까지 거절될 수 있으므로 공식 코드 검색을 먼저 확보합니다.
let result: JapanProductSearchResult | undefined;
let searchError: unknown;
try {
result = await searchProducts(id, 1);
} catch (error) {
searchError = error;
}
try {
return parseProductPage(
await fetchText(`https://jp.daisonet.com/products/${id}`, OPTIONS),
id,
);
} catch (error) {
if (!(error instanceof HttpError) || error.status !== 403) throw error;
const match = result?.products.find((item) => item.id === id);
if (!match) throw searchError ?? error;
const result = await searchProducts(id, 1);
const match = result.products.find((item) => item.id === id);
if (!match) throw error;
return {
...match,
onlineAvailable: null,
Expand Down
5 changes: 3 additions & 2 deletions src/services/daisoJp/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import * as z from 'zod';
import type { ServiceProvider } from '../../core/interfaces.js';
import type { ToolRegistration } from '../../core/types.js';
import type { JapanTransportOptions } from './productRelay.js';
import { searchProducts, getProduct, findStores } from './client.js';

export const jpQuery = z.string().trim().min(1).max(100);
Expand Down Expand Up @@ -42,7 +43,7 @@ const storeOutput = z.object({
brand: z.string(),
});

export function createDaisoJpService(): ServiceProvider {
export function createDaisoJpService(options: JapanTransportOptions = {}): ServiceProvider {
const tools: ToolRegistration[] = [
{
name: 'daiso_jp_search_products',
Expand Down Expand Up @@ -78,7 +79,7 @@ export function createDaisoJpService(): ServiceProvider {
},
handler: async (args) => {
const { id } = schemas.product.parse(args);
const product = await getProduct(id);
const product = await getProduct(id, options);
const result = { ...product };
return {
content: [{ type: 'text', text: JSON.stringify(result) }],
Expand Down
54 changes: 54 additions & 0 deletions src/services/daisoJp/productRelay.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/** 일본 상세만 기존 인증 HTTP 중계에 명시적으로 연결합니다. */
import { ServiceError } from '../../core/errors.js';
import { diagnosticHeaders } from '../../utils/diagnostics.js';
import { parseRelayQuota } from '../../utils/relayQuota.js';
import { dtryxTransportFromBindings, isValidDtryxRelayUrl, type DtryxTransportOptions } from '../dtryx/transport.js';
import { parseProductDetail } from './parser.js';
import type { JapanProduct } from './types.js';
export interface JapanProductRelayOptions extends DtryxTransportOptions { bypassCache?: boolean }
export interface JapanTransportOptions { productRelay?: JapanProductRelayOptions }
export function japanProductTransportFromBindings(bindings?: {
DAISO_JP_PRODUCT_RELAY_ENABLED?: string;
DTRYX_RELAY_URL?: string; DTRYX_RELAY_TOKEN?: string;
DTRYX_ACCESS_CLIENT_ID?: string; DTRYX_ACCESS_CLIENT_SECRET?: string;
}, bypassCache = false): JapanTransportOptions {
return bindings?.DAISO_JP_PRODUCT_RELAY_ENABLED === 'true'
? { productRelay: { ...dtryxTransportFromBindings(bindings), bypassCache } } : {};
}
export async function requestJapanProductRelay(id: string, options: JapanProductRelayOptions): Promise<JapanProduct> {
const { relayUrl, relayToken, accessClientId, accessClientSecret } = options;
if (!isValidDtryxRelayUrl(relayUrl) || !relayToken?.trim() ||
((accessClientId !== undefined || accessClientSecret !== undefined) && (!accessClientId?.trim() || !accessClientSecret?.trim()))) {
throw new ServiceError('DAISO_JP_RELAY_CONFIG_ERROR', '일본 다이소 중계 설정을 확인하세요.', 503, false);
}
const headers: Record<string, string> = { ...diagnosticHeaders(), Authorization: `Bearer ${relayToken}`, 'Content-Type': 'application/json' };
if (options.bypassCache) headers['x-relay-cache-bypass'] = 'true';
if (accessClientId && accessClientSecret) {
headers['CF-Access-Client-Id'] = accessClientId;
headers['CF-Access-Client-Secret'] = accessClientSecret;
}
const controller = new AbortController();
let timer: ReturnType<typeof setTimeout>;
const timeout = new Promise<never>((_, reject) => {
timer = setTimeout(() => { controller.abort(); reject(new ServiceError('DAISO_JP_RELAY_TIMEOUT', '일본 다이소 중계 요청 시간이 초과되었습니다.', 504, true)); }, 15000);
});
try {
const work = (async () => {
const response = await fetch(`${relayUrl!.replace(/\/$/, '')}/v1/daiso-jp/product`, {
method: 'POST', redirect: 'manual', headers, body: JSON.stringify({ id }), signal: controller.signal,
});
if (response.status !== 200) {
void response.body?.cancel().catch(() => undefined);
const status = response.status;
throw new ServiceError('DAISO_JP_RELAY_FAILED', '일본 다이소 중계 요청에 실패했습니다.',
status === 429 || status === 503 || status === 504 ? status : 502,
status === 429 || status >= 500, status, parseRelayQuota(status, response.headers));
}
return parseProductDetail(await response.json(), id);
})();
return await Promise.race([work, timeout]);
} catch (error) {
if (error instanceof ServiceError) throw error;
throw new ServiceError('DAISO_JP_RELAY_FAILED', '일본 다이소 중계 응답을 확인할 수 없습니다.', 502, true);
} finally { clearTimeout(timer!); }
}
27 changes: 27 additions & 0 deletions tests/relay/daiso-jp.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { expect, it, vi } from 'vitest';
import { createJapanProductRelay } from '../../scripts/relay/daiso-jp.js';
const raw = { handle: '4550746075917', title: 'ボウル', price: 22000, available: true };
const req = (body: unknown, token = 'test-token') => new Request('http://localhost/v1/daiso-jp/product', { method: 'POST', headers: { Authorization: `Bearer ${token}` }, body: JSON.stringify(body) });
it('인증된 고정 상품 JSON만 읽고 실제 상품 코드와 대조한다', async () => {
const fetcher = vi.fn(async () => Response.json(raw));
const takeQuota = vi.fn(async () => true);
const handler = createJapanProductRelay('test-token', { fetcher, takeQuota });
expect((await handler(req({ id: raw.handle }, 'wrong'))).status).toBe(401);
for (const body of [null, [], {}, { id: 1 }, { id: '../x' }, { id: raw.handle, url: 'https://other' }]) expect((await handler(req(body))).status).toBe(400);
const response = await handler(req({ id: raw.handle }));
expect(response.status).toBe(200); expect(await response.json()).toEqual(raw);
expect(fetcher.mock.calls[0][0].toString()).toBe('https://jp.daisonet.com/products/4550746075917.js');
expect(takeQuota).toHaveBeenCalledTimes(1);
});
it('원본 차단과 잘못된 상품을 실패로 유지한다', async () => {
for (const response of [new Response('denied', { status: 403 }), new Response(null, { status: 403 }), Response.json({ ...raw, handle: '4550746075918' })]) {
const handler = createJapanProductRelay('test-token', { takeQuota: async () => true, fetcher: vi.fn(async () => response) });
expect((await handler(req({ id: raw.handle }))).status).toBe(502);
}
});

it('원본 실패 본문의 회수 오류를 처리한다', async () => {
const fetcher = vi.fn(async () => ({ status: 403, body: { cancel: async () => { throw Error('cancel failed'); } } }) as unknown as Response);
const handler = createJapanProductRelay('test-token', { takeQuota: async () => true, fetcher });
expect((await handler(req({ id: raw.handle }))).status).toBe(502);
});
8 changes: 6 additions & 2 deletions tests/relay/dtryx-start-cgv.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ it('CGV와 디트릭스 조회가 같은 쿼터를 소비하고 CGV 건강 상
vi.stubEnv('CONVENIENCE_RELAY_TOKEN', undefined);
vi.stubEnv('CONVENIENCE_RELAY_STATE_DIR', undefined);
vi.stubGlobal('fetch', vi.fn().mockImplementation((url: URL | string) => Promise.resolve(
String(url).includes('cgv.co.kr') ? Response.json({ statusCode: 0, data: [] }) : Response.json({ RetCode: 'success', Recordset: [] }),
new URL(String(url)).hostname === 'jp.daisonet.com' ? Response.json({ handle: '4550746075917', title: 'ボウル', price: 22000, available: true }) : ['api.cgv.co.kr', 'cgv.co.kr'].includes(new URL(String(url)).hostname) ? Response.json({ statusCode: 0, data: [] }) : Response.json({ RetCode: 'success', Recordset: [] }),
)));
vi.spyOn(console, 'log').mockImplementation(() => undefined);
vi.spyOn(process, 'once').mockImplementation(((signal: string, listener: () => void) => {
Expand All @@ -52,6 +52,10 @@ it('CGV와 디트릭스 조회가 같은 쿼터를 소비하고 CGV 건강 상
expect((await invoke('/v1/cgv/theaters', {}, 'wrong')).status).toBe(401);
expect((await invoke('/v1/cgv/theaters', {})).status).toBe(200);
expect((await invoke('/v1/dtryx/movies', { brandCode: 'dtryx', cinemaCode: '000067' })).status).toBe(200);
expect((await invoke('/v1/daiso-jp/product', { id: '4550746075917' }, 'wrong')).status).toBe(401);
const japan = await invoke('/v1/daiso-jp/product', { id: '4550746075917' });
expect(japan.status).toBe(200);
expect(japan.body.handle).toBe('4550746075917');
const after = await invoke('/v1/cgv/health');
expect(after.body.quota.dailyRemaining).toBe(before.body.quota.dailyRemaining - 2);
expect(after.body.quota.dailyRemaining).toBe(before.body.quota.dailyRemaining - 3);
});
Loading
Loading
0