Skip to content

suppress Beautiful Soup warnings for URLs during sync_git #20

@Pomax

Description

@Pomax

Any translation field with values that look like URLs currently cause the sync_git command to print warnings to the console like:

INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/day-unite-against-hate-speech-and-misinformation.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/209304/facebook" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/209305/twitter" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/join-mozilla-telling-major-companies-stophateforprofit.po'
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-ethical-ai-the-choice-is-not-yours-sometimes.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/xoue4-ohk1Y?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-how-dating-apps-might-be-keeping-you-single.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/YTlOrapai-M?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-what-is-ai.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/P-iiN0c2uic?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-why-did-i-watch-that.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/pt9YCVX7VOk?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/S0pE87hADUU?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-why-does-youtube-recommend-conspiracy-theory-videos.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/r98HbsaN9nw?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/new-iphone-update-how-to-opt-out-of-cross-app-tracking.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217446/twitter" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217447/facebook" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217448/email" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning

And while true, these warnings are also essentially meaningless, as sync_git is not concerned with validation, it just needs to sync data, whatever that data is. Could the command be updated so that even if it still uses BS4, with loglevel=error rather than what it's using atm?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions