Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Oracle] Add logs #2528

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

[Oracle] Add logs #2528

wants to merge 5 commits into from

Conversation

moxarth-elastic
Copy link
Collaborator

@moxarth-elastic moxarth-elastic commented May 8, 2024

Part Of #2299

Adding more logs in Oracle connector.

Log file: https://drive.google.com/file/d/1eqC6FRfm4IicdMNt2Z1Hivw6x3G-g_JC/view?usp=drive_link

Checklists

Pre-Review Checklist

  • this PR does NOT contain credentials of any kind, such as API keys or username/passwords (double check config.yml.example)
  • this PR has a meaningful title
  • this PR links to all relevant github issues that it fixes or partially addresses
  • if there is no GH issue, please create it. Each PR should have a link to an issue
  • this PR has a thorough description
  • Tested the changes locally
  • Added a label for each target release version (example: v7.13.2, v7.14.0, v8.0.0)

@moxarth-elastic
Copy link
Collaborator Author

buildkite test this

@moxarth-elastic moxarth-elastic mentioned this pull request May 8, 2024
6 tasks
@artem-shelkovnikov
Copy link
Member

@moxarth-elastic example log file that you've attached is actually for OneDrive, can you provide proper log file?

@artem-shelkovnikov
Copy link
Member

The most useful connector logs are - which queries were actually executed by the connectors. Can you add debug logs that state which exact queries were executed? A good addition to it would be to also debug log the number of records returned by the query.

@moxarth-elastic
Copy link
Collaborator Author

@moxarth-elastic example log file that you've attached is actually for OneDrive, can you provide proper log file?

Oops! My bad. I've updated the link.

connectors/sources/oracle.py Outdated Show resolved Hide resolved
connectors/sources/oracle.py Outdated Show resolved Hide resolved
connectors/sources/oracle.py Outdated Show resolved Hide resolved
row_count = await self.oracle_client.get_table_row_count(table=table)
if row_count > 0:
# Query to get the table's primary key
self._logger.debug(f"Total '{row_count}' rows found in '{table}' table")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self._logger.debug(f"Total '{row_count}' rows found in '{table}' table")
self._logger.debug(f"Total {row_count} rows found in table \"{table}\"")

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@artem-shelkovnikov these changes are causing the linting error.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not automatically fixable by make autoformat?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already tried to do that but it was reformatting the log to the previous one. It's not accepting the current logs you suggested.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It cannot reformat to previous one, because I've changed wording of the log slightly?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it complain at \"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it removes the escaping

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do that, it's okay, as long as it's consistent.

moxarth-elastic and others added 2 commits May 21, 2024 16:14
Co-authored-by: Artem Shelkovnikov <artem.shelkovnikov@elastic.co>
@artem-shelkovnikov
Copy link
Member

@moxarth-elastic why did you revert the whole change? The only problem was the escaping? Now I see that log lines are inconsistent - some saytable: {table}, some say '{table}' table.

@moxarth-elastic
Copy link
Collaborator Author

@moxarth-elastic why did you revert the whole change? The only problem was the escaping? Now I see that log lines are inconsistent - some saytable: {table}, some say '{table}' table.

Oh, I see. Let me update it.

@moxarth-elastic
Copy link
Collaborator Author

@artem-shelkovnikov I've made the logs consistent and fixed the linter error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants