From approximately 6:00 AM (JST) on March 18, 2025 to 7:40 PM (JST) on March 18, 2025.
Incident Details:
The Webhook feature in kintone is designed not to trigger when multiple records are added or edited in bulk via the REST API. However, when calling records.json using the REST API with UPSERT mode turned off (default behavior), an unintended Webhook trigger was occurring.
As a result, for users who had set up Gusuku Customine's Job Runner to receive Webhook triggers from kintone, the Webhook was unintentionally activated multiple times. This led to a massive number of Job Runner jobs being initiated, causing a significant depletion of the allocated execution time for Job Runner.
Cause of the Incident:
The issue was caused by an internal update to kintone performed by Cybozu in the early morning of March 18, 2025.
For further details, please refer to Cybozu’s official announcement:
At approximately 7:40 PM (JST) on March 18, 2025, Cybozu completed the rollback of the faulty program, and the issue was resolved.
Our Response
(1) Response for All Customers
Considering that some customers unintentionally consumed their Job Runner execution time, reducing their remaining time due to this issue, we have modified the Job Runner settings as of 6:20 PM (JST) on March 18, 2025. Now, all customers using Job Runner can execute jobs even if their remaining execution time is fully consumed.
This temporary measure will remain in effect until 12:00 AM (JST) on March 31, 2025. Starting April 1, 2025, the normal execution limits based on the allocated application slots for Job Runner will be reinstated.
Even during this temporary period, administrator users will still receive a notification email when remaining execution time falls below 20%. However, this email can be ignored until March 31, 2025.
[Action Required from Customers]
If you increased your application slot allocation for Job Runner due to this issue, you may now reduce the additional slots at your discretion. Even if the additional application slot execution time has already been used, it is still possible to decrease the allocated slots.
However, please complete this reduction process by March 31, 2025.
(2) Response for Customers Who Purchased Additional Application Slots under the Monthly Plan
[Action Required from Customers]
If you purchased additional application slots specifically to address this incident, please contact our email support (butler@gusuku.io) with the following details by 12:00 AM (JST) on March 25, 2025:
The kintone subdomain where you are using Customine
The approximate date and time when the additional application slots were purchased
The number of additional application slots purchased
The number of additional application slots allocated to Job Runner
We will review the additional application slot purchase and allocation status, and we will proceed with cancelling the additional purchase. As a result, you will not be charged for the extra application slots.
Please note: If you contact us after the deadline (March 25, 2025, at 12:00 AM JST), we may not be able to process your request. Thank you for your understanding.
次の場所のその他のトピック：
Maintenance and incident announcement / 障害・メンテナンス情報
[R3] Support
■ 障害の概要
発生日時:
2025年3月18日 6:00(JST)頃～2025年3月18日 19:40(JST)まで
障害内容:
kintoneのWebhookは、REST APIで複数のレコードを一括追加・編集した場合には発生しない仕様ですが、REST APIで、records.json を呼び出す際に、UPSERTモードをオフ（デフォルトの挙動）で呼び出した場合に、意図せずWebhookが発生していました。
これにより、gusuku CustomineのJob RunnerでkintoneからのWebhookを受信する設定をされている場合、意図せずJob Runnerのジョブが大量に起動されてしまい、Job Runnerの実行割り当て時間が大量消費されてしまう事象が発生しました。
障害原因:
2025年3月18日早朝にサイボウズ社によって行われたkintoneの内部アップデートによる不具合です。
詳細はサイボウズ社の発表を参照してください。
【復旧】「kintone」の障害について
障害への根本対応:
2025年3月18日19:40(JST)頃にサイボウズ社により不具合のあるプログラムからの切り戻し作業が行われ復旧しました。
■ 当社の対応
（1）すべてのお客様への対応
本事象により意図せずJob Runnerの実行時間を消費し残時間が減ってしまったお客様が発生していることを鑑み、2025年3月18日18:20(JST)時点で、Job Runnerをご利用のすべてのお客様で残時間をすべて消費してもジョブを実行するように変更しました。この暫定対応は2025年3月31日24:00(JST)まで継続します。2025年4月1日からは通常のJob Runnerに割り当てていただいたアプリスロット数にもとづく制限に戻ります。
この暫定対応期間中もオーナー権限を持つgusukuユーザーの方には、Job Runnerに割り当てられている時間の残りが20%を切った場合、その旨を伝えるメールが送信されますが、2025年3月31日いっぱいまでは本メールを無視していただいて問題ありません。
【お客様の対応が必要】
本事象への対応のためにJob Runnerへのアプリスロットの割り当てを追加されたお客様は、本発表以降追加でJob Runnerに割り当てられたアプリスロットを減数していただくことが可能です。追加されたアプリスロット分の実行時間を消費していても減数できるようにしております。なお、この減数作業は2025年3月31日中までに完了してください。
（2）月額プランをご利用で追加のアプリスロットを購入されたお客様への対応
【お客様の対応が必要】
本障害に対応するためにアプリスロットを追加購入いただいたお客様は個別にメールサポート（ butler@gusuku.io )へ以下の情報を2025年3月25日24:00(JST)までにご連絡ください。
・カスタマインをご利用になっているkintoneのサブドメイン
・アプリスロットを追加購入された日時（およそで構いません）
・追加購入されたアプリスロットの数
・追加でJob Runnerに割当されたアプリスロットの数
アプリスロットの追加状況と、Job Runnerへの割当状況を確認させていただいた上で、アプリスロットの追加をキャンセルさせていただきます。この対応により、追加で購入いただいたアプリスロットへの課金は行われません。
なお、2025年3月25日24:00(JST)の期限を過ぎてご連絡いただいた場合は、対応いたしかねる場合がありますのでご了承ください。
Incident Summary
Occurrence Date and Time:
From approximately 6:00 AM (JST) on March 18, 2025 to 7:40 PM (JST) on March 18, 2025.
Incident Details:
The Webhook feature in kintone is designed not to trigger when multiple records are added or edited in bulk via the REST API. However, when calling
records.jsonusing the REST API with UPSERT mode turned off (default behavior), an unintended Webhook trigger was occurring.As a result, for users who had set up Gusuku Customine's Job Runner to receive Webhook triggers from kintone, the Webhook was unintentionally activated multiple times. This led to a massive number of Job Runner jobs being initiated, causing a significant depletion of the allocated execution time for Job Runner.
Cause of the Incident:
The issue was caused by an internal update to kintone performed by Cybozu in the early morning of March 18, 2025.
For further details, please refer to Cybozu’s official announcement:
[Resolved] kintone Service Disruption
Permanent Resolution of the Incident:
At approximately 7:40 PM (JST) on March 18, 2025, Cybozu completed the rollback of the faulty program, and the issue was resolved.
Our Response
(1) Response for All Customers
Considering that some customers unintentionally consumed their Job Runner execution time, reducing their remaining time due to this issue, we have modified the Job Runner settings as of 6:20 PM (JST) on March 18, 2025. Now, all customers using Job Runner can execute jobs even if their remaining execution time is fully consumed.
This temporary measure will remain in effect until 12:00 AM (JST) on March 31, 2025.
Starting April 1, 2025, the normal execution limits based on the allocated application slots for Job Runner will be reinstated.
Even during this temporary period, administrator users will still receive a notification email when remaining execution time falls below 20%. However, this email can be ignored until March 31, 2025.
[Action Required from Customers]
If you increased your application slot allocation for Job Runner due to this issue, you may now reduce the additional slots at your discretion.
Even if the additional application slot execution time has already been used, it is still possible to decrease the allocated slots.
However, please complete this reduction process by March 31, 2025.
(2) Response for Customers Who Purchased Additional Application Slots under the Monthly Plan
[Action Required from Customers]
If you purchased additional application slots specifically to address this incident, please contact our email support (butler@gusuku.io) with the following details by 12:00 AM (JST) on March 25, 2025:
We will review the additional application slot purchase and allocation status, and we will proceed with cancelling the additional purchase.
As a result, you will not be charged for the extra application slots.
Please note: If you contact us after the deadline (March 25, 2025, at 12:00 AM JST), we may not be able to process your request. Thank you for your understanding.