Blow their own horns HN: StreamPQ – Circulation PostgreSQL query results in Python with out a cursor

Circulation results of multi-commentary PostgreSQL queries from Python with out server-facet cursors. Has advantages over any other Python PostgreSQL libraries:

• Streams results from complex multi-commentary queries despite the undeniable truth that SQL would not permit server-facet cursors for such queries – merely for neat quantities of results that invent now not slot in reminiscence.

• CTRL+C (SIGINT) by default behaves as expected even for the length of slack queries – a KeyboardInterrupt is raised and swiftly bubbles up thru streampq code. Until shopper code prevents it, the program will exit.

• Each effort is made to execute queries on KeyboardInterrupt, SystemExit, or errors – the server would not proceed needlessly using resources.

In particular fine when non permanent tables are needed to store intermediate results in multi-commentary SQL scripts.

Installation

The libpq binary library is also required. Right here is on occasion either already keep in, or keep in by:

• macOS + brew: brew set up libpq
• Linux (Debian): unprejudiced correct set up libpq5
• Linux (Red Hat): yum set up postgresql-libs

Utilization

from streampq import streampq_connect

# libpq connection paramters
# https://www.postgresql.org/clinical doctors/fresh/libpq-connect.html#LIBPQ-PARAMKEYWORDS
#
# Any could presumably well furthermore furthermore be ommitted and ambiance variables will seemingly be damaged-down as a replacement
# https://www.postgresql.org/clinical doctors/fresh/libpq-envars.html
connection_params = (
    ('host', 'localhost'),
    ('port', '5432'),
    ('dbname', 'postgres'),
    ('person', 'postgres'),
    ('password', 'password'),
)

# SQL commentary(s) - if bigger than one, separate by ;
sql = '''
SELECT FROM my_table WHERE my_col={first};
SELECT FROM my_other_table WHERE my_other_col={second};
'''

with streampq_connect(connection_params) as query:
    for (columns, rows) in query(sql, literals=(
        ('first', 'a cost'),
        ('second', 'one other price'),
    )):
        for row in rows:
            crawl

Exceptions

Exceptions salvage from streampq.StreamPQError. If there is from now on files available on the error, or now not it is added as a string in its args property. Right here is integrated in the string illustration of the exception by default.

Exception hierarchy

• StreamPQError

  Unsuitable class for all explicitly-thrown exceptions

  • ConnectionError

    An error came about whereas making an try to connect to the database.

  • QueryError

    An error came about whereas making an try to bustle a query. Most incessantly here is ensuing from a syntax error or a missing column.

  • CancelError

    An error came about whereas making an try to execute a query.

  • CommunicationError

    An error came about talking with the database after a success connection.