o KDiUE@sddlmZddlZddlZddlmZddlmZddlm Z m Z m Z ddl m Z ddlZddlmZddlZddlZed Zed d ZGd d d eZGdddeZGdddeZddeeeefDZdS))unicode_literalsN) namedtuple) ModelBase)escape parse_tsvimport_submodules)ceil)Templateclickhouse_ormPagez6objects number_of_objects pages_total number page_sizec@seZdZdZdS)DatabaseExceptionz1 Raised when a database operation fails. N)__name__ __module__ __qualname____doc__rrU/var/www/Datamplify/venv/lib/python3.10/site-packages/infi/clickhouse_orm/database.pyr sr csneZdZdZfddZedejejBedejejBedejejBfZ e ddZ d d Z Z S) ServerErrorz0 Raised when a server returns an error. cs>d|_||}|r|\|_|_dS||_tt||dSN)codeget_error_code_msgmessagesuperr__init__)selfr processed __class__rrrs  zServerError.__init__z Code:\ (?P\d+), \ e\.displayText\(\)\ =\ (?P[^ \n]+):\ (?P.+?), \ e.what\(\)\ =\ (?P[^ \n]+) zq Code:\ (?P\d+), \ e\.displayText\(\)\ =\ (?P[^ \n]+):\ (?P.+) zZ Code:\ (?P\d+). \ (?P[^ \n]+):\ (?P.+) cCsB|jD]}||}|rt|d|dfSqd|fS)z Extract the code and message of the exception that clickhouse-server generated. See the list of error codes here: https://github.com/yandex/ClickHouse/blob/master/dbms/src/Common/ErrorCodes.cpp rmsgr)ERROR_PATTERNSmatchintgroupstrip)clsfull_error_messagepatternr!rrrr=s   zServerError.get_error_code_msgcCs|jdur d|j|jSdS)Nz{} ({}))rformatrrrrr__str__Ms zServerError.__str__)rrrrrrecompileVERBOSEDOTALLr classmethodrr* __classcell__rrrrrs      rc@seZdZdZ   d7ddZd d Zd d Zd dZddZddZ d8ddZ ddZ d9ddZ d:ddZ d;ddZdd%d&Zd'd(Zdsz0Database.get_model_for_table..T)r7rU iter_linesrcreate_ad_hoc_model_system _readonly)rrb system_tabler7rdlinesfieldsmodelrrrget_model_for_tables   zDatabase.get_model_for_tablecCs>t|ts Jd|dur|j|ddSt||j|<dS)aU Adds a database setting that will be sent with every request. For example, `db.add_setting("max_execution_time", 10)` will limit query execution time to 10 seconds. The name must be string, and the value is converted to string in case it isn't. To remove a setting, pass `None` as the value. zSetting name must be a stringN) isinstancestrrApop)rnamevaluerrr add_settingszDatabase.add_settingcsddlmt|ztWn tyYdSwjs&r*tdd ddj dd D} r>d nd }d ||ffd d} |dS)z Insert records into the database. - `model_instances`: any iterable containing instances of a single model class. - `batch_size`: number of records to send per chunk (use a lower number if your records are very large). r)BytesIONz1You can't insert into read only and system tables,cSsg|]}d|qS)`%s`r)rirxrrrrksz#Database.insert..T)writableTSKV TabSeparatedz"INSERT INTO $table (%s) FORMAT %s c3s}|d|d}D] }||||d7}|kr@|V}d}q |rJ|VdSdS)Nutf-8rhrr)write _substituteencode set_database to_db_stringgetvalue)bufrqinstancer| batch_sizefirst_instanceir]queryrrrgens$   zDatabase.insert..gen) ior|iternext StopIterationr is_read_onlyrYr joinrrhas_funcs_as_defaultsrU)rmodel_instancesr fields_listfmtrrrrinserts"    zDatabase.insertcCsbddlm}d}|rt||r||}|dt|7}|||}||}|jr/t|jSdS)z Counts the number of records in the model's table. - `model_class`: the model to count. - `conditions`: optional SQL conditions (contents of the WHERE clause). rQzSELECT count() FROM $table WHERE ) infi.clickhouse_orm.queryrruto_sqlrvrrUrcr")rr] conditionsrrrerrrcounts     zDatabase.countc cs~|d7}|||}|||d}|}tt|}tt|}|p+tt||}|D]}|r<||||j |Vq.dS)aH Performs a query and returns a generator of model instances. - `query`: the SQL query to execute. - `model_class`: the model class matching the query's table, or `None` for getting back instances of an ad-hoc model. - `settings`: query settings to send as HTTP GET parameters z% FORMAT TabSeparatedWithNamesAndTypesTN) rrUrlrrrrmzipfrom_tsvrL) rrr]rArerq field_names field_typesrjrrrselect s   zDatabase.selectcCs||d}|j|||djS)a Performs a query and returns its output as text. - `query`: the SQL query to execute. - `settings`: query settings to send as HTTP GET parameters - `stream`: if true, the HTTP response from ClickHouse will be streamed. N)rAstream)rrUrc)rrrArrrrraw s z Database.rawrdc Csddlm}|||}tt|t|} |dkr t| d}n |dkr*td||d|} dd | } |rPt ||rH| |}| dt|7} | d |7} | d | |f7} || |} t|rpt|| ||ng|| ||d S) a Selects records and returns a single page of model instances. - `model_class`: the model class matching the query's table, or `None` for getting back instances of an ad-hoc model. - `order_by`: columns to use for sorting the query (contents of the ORDER BY clause). - `page_num`: the page number (1-based), or -1 to get the last page. - `page_size`: number of records to return per page. - `conditions`: optional SQL conditions (contents of the WHERE clause). - `settings`: query settings to send as HTTP GET parameters The result is a namedtuple containing `objects` (list), `number_of_objects`, `pages_total`, `number` (of the current page), and `page_size`. rrrzInvalid page number: %dzSELECT {} FROM $tablez, rz ORDER BY %sz LIMIT %d, %d)objectsnumber_of_objects pages_totalnumber page_size)rrrr"r floatmax ValueErrorr(rrrkeysrurrvrr listr) rr]order_bypage_numrrrArrroffsetrrrrpaginate+s.         zDatabase.paginate'c Csddlm}td}||}t|}t||}t|D]0}| d|||j D]} | |q-| |||t jdgt|dd|krPdSq dS)z Executes schema migrations. - `migrations_package_name` - fully qualified name of the Python package containing the migrations. - `up_to` - number of the last migration to apply. rMigrationHistory migrationszApplying migration %s...) package_name module_nameappliedN)rrlogging getLogger_get_applied_migrationsrsetrsortedinfo operationsapplyrdatetimedatetodayr") rmigrations_package_nameup_torloggerapplied_migrationsmodulesunapplied_migrationsrx operationrrrmigrateSs      zDatabase.migratecCsBddlm}||d|}|||}tdd||DS)Nrrz8SELECT module_name from $table WHERE package_name = '%s'css|]}|jVqdSr)r)riobjrrr msz3Database._get_applied_migrations..)rrr^rrr)rrrrrrrrhs   z Database._get_applied_migrationscCs`t|tr|d}|jrt|||}|jj|j ||||j d}|j dkr.t |j |S)Nr)paramsdatarr:)rurvrr@rr _build_paramsr=postr8r: status_coderrc)rrrArrrerrrrUos      zDatabase._sendcCs@t|pi}||j|jr|j|d<|jr|jsd|d<|S)Ndatabaserar9)dictupdaterArBr7r9rE)rrArrrrrzs    zDatabase._build_paramscCs^d|vr-td|jd}|r&|rd||d<n d|j|f|d<t||}|S)zD Replaces $db and $table placeholders in the query. $r~)dbz `system`.`%s`tablez `%s`.`%s`)rr7rYrbr safe_substitute)rrr]mappingrrrrszDatabase._substitutec CsTz|d}t|jWSty)}ztd|tjWYd}~Sd}~ww)NzSELECT timezone()z3Cannot determine server timezone (%s), assuming UTC) rUrJtimezonercr$rr exceptionrK)rreerrrrIs  zDatabase._get_server_timezonec Csjz |d}|j}Wnty$}ztd|d}WYd}~nd}~ww|r3tdd|dDS|S)NzSELECT version();z4Cannot determine server version (%s), assuming 1.1.0z1.1.0css |] }|rt|VqdSr)isdigitr")rinrrrrsz/Database._get_server_version...)rUrcrrrtuplesplit)ras_tuplereverrrrrrGs    zDatabase._get_server_versioncCs|d|j}|jdkS)Nz6SELECT count() FROM system.databases WHERE name = '%s'ra)rUr7rcr$rrerrrrCszDatabase._is_existing_databasecCs|d}|jdkS)Nz9SELECT value FROM system.settings WHERE name = 'readonly'0)rUrcr$rrrrrDs z Database._is_connection_readonly)r2NNFTr3TF)F)r{r)NN)NF)rrNN)r)T)rrrrrrFrWr^r`rfrtrzrrrrrrrrUrrrIrGrCrDrrrrr1Rs6 ,    -   (    r1cCsg|]}|jqSr)r)ricrrrrksrk) __future__rr+r; collectionsrmodelsrutilsrrrmathr rstringr rJrrrr Exceptionr robjectr1__all__rrrrs&       7]