[CAL-3752] Unreliable Triggering and Payload Inconsistencies in Calcom Webhooks

[hussamkhatib]

Issue Summary

My team has been facing issues with calcom webhooks one after the other. The triggering of the webhooks isn't reliable, so is the payload of the triggers

MEETING_STARTED and MEETING_ENDED

• Triggers at incorrect time (For eg both meeting start and end was triggered at 4:54:13)

• Sometimes it does not trigger at all. (It might be the case when the booking is booked very close to the current time)

• Irregular Request body (main issue)

  • 1st variant

 {
       triggerEvent: 'MEETING_STARTED',
       id: 2030837,
       uid: 'b7bUM9yNBpURfEtRy5fSYM',
       idempotencyKey: 'c86af8f1-2d63-5a04-ad8d-d209fc691a56',
       userId: 64419,
       userPrimaryEmail: 'hussamxxxxxx@gmail.com',
       eventTypeId: 788720,
       title: 'test between Hussam Khatib and Hussam',
       description: '',
       customInputs: {},
       responses: {
         name: 'Hussam',
         email: 'hussam@xxxx.com',
         guests: [],
         location: [Object]
       },
       startTime: '2024-05-14T18:49:02.000Z',
       endTime: '2024-05-14T18:50:02.000Z',
       location: 'integrations:daily',
       createdAt: '2024-05-14T18:48:07.759Z',
       updatedAt: null,
       status: 'ACCEPTED',
       paid: false,
       destinationCalendarId: null,
       cancellationReason: null,
       rejectionReason: null,
       dynamicEventSlugRef: null,
       dynamicGroupSlugRef: null,
       rescheduled: null,
       fromReschedule: 'hR5Lzz7BmE1s1erY6Zd5at',
       recurringEventId: null,
       smsReminderNumber: null,
       scheduledJobs: [],
       metadata: {
         videoCallUrl: 'https://app.cal.com/video/xxxxxxxx'
       },
       isRecorded: false,
       iCalUID: 'hR5Lzz7BmE1s1erY6Zd5at@Cal.com',
       iCalSequence: 0,
       rating: null,
       ratingFeedback: null,
       noShowHost: null,
       user: {
         email: 'hussamxxxxx@gmail.com',
         name: 'Hussam Khatib',
         timeZone: 'Asia/Calcutta',
         username: 'hussamkhatib'
       },
       attendees: [ [Object] ],
       payment: [],
       references: [],
       appsStatus: [ [Object] ]
     }
   }

• I had opened an issue for it, we had decided to move ahead with handling these 2 triggerEvent differently only to find out that something attendees is not defined.
• There's no organizer Object.

RECORDING_READY

• Does not trigger at all. (Note: It's been only few days since we are facing this Issue, it used to work fine a few days back)
• Also has a different payload.

export interface RECORDING_READY {
  type: string
  title: string
  startTime: string
  endTime: string
  organizer: Organizer
  attendees?: AttendeesEntity[] | null
  uid: string
  downloadLink: string
}

Steps to Reproduce

Follow the steps mentioned in the docs to setup webhooks and print logs of the request body with timestamps
Use the link provided in the meeting(CAL video) and record the meeting

Actual Results

• You will find no logs printed for RECORDING_READY webhook
• You should find a different payload for MEETING_STARTED and MEETING_ENDED than what is mentioned in the docs.

Evidence

• The snippets above are Response of actual booking made app.cal.com

_CAL-3752

[dosubot]

Hey @hussamkhatib, great to see you back! 😀 We definitely remember working through some challenges together. I'm diving into the details of what you've shared and will circle back with a comprehensive response soon.

[l0g1x]

Additionally, the automated emails the calcom would send to all attendees and the organizer once the recording is completed, no longer get sent

[dosubot]

To address the issues you're encountering with Cal.com webhooks, let's dive into specifics:

1. MEETING_STARTED and MEETING_ENDED Events Timing and Triggering:

   • The Cal.com system schedules triggers for MEETING_STARTED and MEETING_ENDED events based on the booking's start and end times, ensuring that events are triggered as expected, even for bookings made close to the current time. This process is managed through the scheduleTrigger function, which prepares and schedules the event triggers in the database.

2. Payload Inconsistencies:

   • The structure of payloads for MEETING_STARTED, MEETING_ENDED, and RECORDING_READY events might vary. It's important to review recent changes or issues in the Cal.com repository that could affect payload structures. For example, discussions in issues like [CAL-2281], MEETING_ENDED webhook payload shape differs, and Not receiving Meeting Ended webhook might provide insights into similar problems and potential solutions.

3. RECORDING_READY Event and Automated Emails:

   • The RECORDING_READY event and the process for sending automated emails upon recording completion involve initializing classes with event and attendee details to prepare email payloads. Recent changes, such as the addition of RECORDING_READY to the WebhookTriggerEvents enum, could impact the triggering of this event and the email sending process. It's crucial to ensure that the application's logic is updated to handle this new event type properly.

For further assistance, please share more details about your Cal.com codebase version, the steps you're following to set up and trigger the webhooks, and any specific error messages or logs. This information will be vital for diagnosing and resolving the issues more effectively.

Details

Sources

• [CAL-2281] payload and headers wrong for meeting ended webhook trigger
• MEETING_ENDED webhook payload shape differs from all other webhook types
• Not receiving Meeting Ended webhook
• apps/web/pages/api/recorded-daily-video.ts
• packages/features/webhooks/lib/sendPayload.ts
• packages/emails/templates/attendee-daily-video-download-recording-email.ts
• packages/emails/templates/organizer-daily-video-download-recording-email.ts
• packages/features/bookings/lib/handleNewBooking.ts
• packages/features/webhooks/lib/scheduleTrigger.ts
• packages/prisma/migrations/20230414112451_add_recording_ready/migration.sql
• packages/features/webhooks/lib/constants.ts

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[CarinaWolli]

Triggers at incorrect time (For eg both meeting start and end was triggered at 4:54:13)

Our meeting ended and started trigger doesn't trigger at the exact time as we have a corn job running every 5 minute that then triggers all webhooks. Your meeting starts at 4:50 and ends at 4:51, that's why both were triggered at the same time at 4:54

Sometimes it does not trigger at all. (It might be the case when the booking is booked very close to the current time)

That can happen when the trigger time is closer than 5 minutes to the current time. Also there was a current fix for another issue related to that #14864, however old bookings that were created before the fix can still cause the issue

Irregular Request body (main issue)

We are currently working on fixing the request body of the meeting started and meeting ended trigger #15036

RECORDING_READY
Does not trigger at all. (Note: It's been only few days since we are facing this Issue, it used to work fine a few days back)

Tagging @Udit-takkar for this one

---

@hussamkhatib Let me know if I didn't address everything and if you have any questions to that

[PeerRich]

added highest priority

[hussamkhatib]

Thanks @CarinaWolli, that’s helpful!

[Udit-takkar]

RECORDING_READY
Does not trigger at all. (Note: It's been only few days since we are facing this Issue, it used to work fine a few days back)

This was fixed yesterday.

[l0g1x]

We can’t figure out which PR you are referring to, mind linking it?

…

On Fri, May 17, 2024 at 5:14 AM Udit Takkar ***@***.***> wrote: RECORDING_READY Does not trigger at all. (Note: It's been only few days since we are facing this Issue, it used to work fine a few days back) This was fixed yesterday. — Reply to this email directly, view it on GitHub <#15078 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJ4JMSPRB77FD5EOE6RDVTZCXJ6RAVCNFSM6AAAAABH2KLRFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJXGIZDKMZQGQ> . You are receiving this because you commented.Message ID: <calcom/cal. ***@***.***>

[Udit-takkar]

@l0g1x No PR was required for this. I had to make a POST request to restart subscribing to recording ready event.

[hussamkhatib]

RECORDING_READY
Does not trigger at all. (Note: It's been only few days since we are facing this Issue, it used to work fine a few days back)

This was fixed yesterday.

Thanks @Udit-takkar, It works now

[hussamkhatib]

@Udit-takkar We had a Meeting an hour back, the recording ready din't trigger again.

[Udit-takkar]

@Udit-takkar We had a Meeting an hour back, the recording ready din't trigger again.

Restarted this again. investigating why it's failing sometimes