K!‘Dn S (dZddlZddlZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddl m Z ddl mZddl mZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZe eeeeeeefZeje Z!ej"hZ#GddZ$GddZ%d dZ&dedeej'j(fdZ)dededefdZ*de%deeee eefffdZ+dedej,deeeffdZ-dS)!z$Checker Manager and Checker classes.N)Any)Dict)List)Optional)Tuple)defaults) exceptions) processor)utils) expand_paths)Checkers) LoadedPlugin)StyleGuideManagerceZdZdZdededdfdZddZdefdZ d e d e defd Z dd e ee ddfd ZdeeeffdZddZddZddZdd e ee ddfdZddZdS)ManageraRManage the parallelism and checker instances for each plugin and file. This class will be responsible for the following: - Determining the parallelism of Flake8, e.g.: * Do we use :mod:`multiprocessing` or is it unavailable? * Do we automatically decide on the number of jobs to use or did the user provide that? - Falling back to a serial way of processing files if we run into an OSError related to :mod:`multiprocessing` - Organizing the results of each checker so we can group the output together and make our output deterministic. style_guidepluginsreturnNc||_|j|_||_||_g|_g|_ddddd|_ttj |jj |jj |_ dS)z Initialize our Manager instance.r)files logical linesphysical linestokensN) roptionsr _job_countjobs _all_checkerscheckers statisticstuple itertoolschainexcludeextend_exclude)selfrrs /srv/buildsys-work-dir/castor/build_node/builder-2/WGSG1/unpkd_srcs/cloudlinux-venv-1.0.6/venv/lib/python3.11/site-packages/flake8/checker.py__init__zManager.__init__@s '"*  OO%% 02+-      ODL0$,2M N N   c|jD]1}tjD]"}|j|xx|j|z cc<#2|jdxxt |jz cc<dS)Nr)rrSTATISTIC_NAMESrlen)r%checker statistics r&_process_statisticszManager._process_statisticsVs} L LG%5 L L  ***g.@.KK**** L    C $6$66     r(ctjdkrtddSt j|jjrtddS|jjrtddS|jj }|j r% tj S#t$rYdSwxYw|j S)NforkzGThe multiprocessing module is not available. Ignoring --jobs arguments.rz]The --jobs option is not compatible with supplying input using - . Ignoring --jobs arguments.zcThe --diff option was specified with --jobs but they are not compatible. Ignoring --jobs arguments.)multiprocessingget_start_methodLOGwarningr is_using_stdinr filenamesdiffris_auto cpu_countNotImplementedErrorn_jobs)r%rs r&rzManager._job_count\s  + - - 7 7 KK-   1   6 7 7  KK=   1 <   KKF   1|  <  &0222&   qq  {s*B== C  C filenameresultsc j|j}d}|D]&\}}}}} |||||||| z }'|S)Nr)coder< line_number column_numbertext physical_line)r handle_error) r%r<r=rreported_results_count error_coder@columnrBrCs r&_handle_resultszManager._handle_resultssi& !"FM   BZfdM "k&>&>!'$+ '?''  " "&%r(pathsc:| jj}fdt|jjjjjjjD_djD_t dtjdS)zCreate checkers for each file.NcHg|]}t|jjS))r<rr) FileCheckerrr).0r<r%s r& z)Manager.make_checkers..sG     !         r()rIstdin_display_namefilename_patternsr#is_running_from_diffc g|] }|j | S)should_process)rMcs r&rNz)Manager.make_checkers..s KKKq!:JKKKKr(zChecking %d files) rr6r rOr<r#r7rrr3infor+r%rIs` r& make_checkerszManager.make_checkerss =L*E     )#'<#B"&,"7 %)\%6     LKD$6KKK  $c$-&8&899999r(cdx}}|jD]}}t|jd}|j}|j|5||||z }dddn #1swxYwY|t|z }~||fS)aReport all of the errors found in the managed file checkers. This iterates over each of the checkers and reports the errors sorted by line number. :returns: A tuple of the total results found and the results reported. rc"|d|dfS)NrS)tups r&z Manager.report..ss1vs1v>Nr()keyN)rsortedr= display_namerprocessing_filerHr+)r%results_reported results_foundr,r=r<s r&reportzManager.reports,-,=) * *GW_2N2NOOOG+H!11(;; L L D$8$87$K$KK  L L L L L L L L L L L L L L L S\\ )MM/00sA,,A0 3A0 c tjt}tjt}t |j}||dSd} |t|j tt|j |j}|D]}|\}}} |||<| ||<| | d}|s(|| n0#|s)|| wwxYw|j D]#} | j}||| _||| _$dS)zRun the checkers in parallel.NF) chunksizeT) collections defaultdictlistdict_try_initialize_processpoolr run_serialimap_unordered _run_checksrcalculate_pool_chunksizer+closejoin terminaterar=r) r% final_resultsfinal_statisticspool pool_closedpool_mapretr<r=rr,s r& run_parallelzManager.run_parallelsT_SjkoSpSp 6A6Md6S6S+4955 < OO    F  ** 2 && +H  8 803-':*1 h'-7 ** JJLLL IIKKKK           } < % $ 7D &*n&G&G&I&I"ID 03DN4H0I0IDO, - - - & %r(cd|jS)z)Provide helpful debugging representation.zFileChecker for )r<rs r&__repr__zFileChecker.__repr__,s1$-111r(c  tj|j|jS#t$r:}|dddt |jd|Yd}~dSd}~wwxYw)NE902r: )r FileProcessorr<rOSErrorretyper)r%es r&rzFileChecker._make_processor0s{ *4=$,GG G    KK1a)9&@&@Q&@&@ A A A44444 s! A%/A  A%rFr@rGrBc||dd\}}t|dr"|j|j|}nd}|j|||||f|S)z2Report an error by storing it in the results list.N r[r )splithasattrr noqa_line_forr=append)r%rFr@rGrBlines r&rezFileChecker.report=s  #zz#q11 J 4 % % $.*D>// <> ZZ!^ $Qx &),&7&7#///0F : C m #FF{r(c,|jJ|j}|jjD]d}|||} |}n#t $r|}YnwxYw|D] \}}}}|d|||!edS)z1Run all checks expecting an abstract syntax tree.N)treerFr@rGrB)r build_astrrrrrre) r%astrr,runnerr@offsetrB_s r&run_ast_checkszFileChecker.run_ast_checkss~)))n&&((l'  FnnV#n66G ! ! ! ! !  !28  .fdA # +!    s A A-,A-c\|jJ|j\}}}|sdS|j|td||jjD]}|j|| ||pd}|D]\\}}t||\}} || cxkrdkrnnt d|| d|| |]|j dS)z(Run all checks expecting a logical line.NzLogical line: "%s")rrSrz#position of error out of bounds: %sr)r build_logical_line update_stater3debugrrrupdate_checker_state_forr find_offsetr4renext_logical_line) r%commentsrmappingrr=rrBr@rs r&run_logical_checkszFileChecker.run_logical_checkss]~)))*..*K*K*M*M',  F ##G,,, & (;(;(=(=>>>l/  F N 3 3F ; ; ;nnV,nGGM2G '   -8-I-I* ]-4444144444KK EvNNN # +(   ((*****r(rCcl|jJ|jjD]}|j||||}|hd} |d}n#t t f$rYnwxYwt|tr|f}|D]*}|\}}| d|jj ||+dS)zoRun all checks for a given physical line. A single physical check may return multiple errors. N)rCrr) r rrCrr IndexError TypeErrorrrrer@)r%rCrresultr result_singlerBs r&run_physical_checkszFileChecker.run_physical_checkss ~)))l0  F N 3 3F ; ; ;^^F-^HHF! $ $*1IMM"I.DmS11'$YF%+M*7'M4KK#'$(N$>,!  %  sAA+*A+c|jJd}|j}|j}d}|D]}|dxxdz cc<||||dd\}}|tjkrtj||}n/|dkr)tj|r|||d}|j r6| |j d| dSdS) zProcess tokens and trigger checks. Instead of using this directly, you should use :meth:`flake8.checker.FileChecker.run_checks`. Nrrr[r\r) r rgenerate_tokenscheck_physical_eolrOPcount_parenthesestoken_is_newlinehandle_newlinerrrr)r%parensrfile_processor prev_physicalr token_typerBs r&process_tokenszFileChecker.process_tokenss. ~)))_  #3355 % %E x A %  # #E= 9 9 9$QqSz JX[(("4VTBB1-e444'' 333!!HMM   &  $ $^%9"%= > > >  # # % % % % % & &r(c |jJ ||n#ttjf$r}t |tjrdnd}||\}}||||t|j d|j d|j |j |jfcYd}~Sd}~wwxYw|jjd}||jd<|j |j |jfS)zRun checks against the file.NrE999rrr)r rrrrrrrrerrrr<r=r)r%rr?rrG logical_liness r&r|zFileChecker.run_checkss!~))) @    ! ! !    ! ! ! !X01 @ @ @'8+>??K66VD::1==KC KKc6d1gg.>+M+M!&)+M+M N N N=$,? ? ? ? ? ? ?  @ 1/B +8(}dlDO;;s(4C BC CCrc`|jJ|tjkr/||jdSt |jjdkr4|j|jdS|dS)z3Handle the logic when encountering a newline token.Nr[) r rNEWLINErreset_blank_beforer+rvisited_new_blank_linedelete_first_token)r%rs r&rzFileChecker.handle_newlines~))) ) ) )  # # % % % N - - / / / / / & ' '1 , , N 1 1 3 3 3 N - - / / / / /  # # % % % % %r(rrc|jJtj|r@|ddkr||dS||ddStj|rw|dd}|j|5|j|D]}|| ddddS#1swxYwYdSdS)z@Run physical checks if and only if it is at the end of the line.Nrrr\r)r@)r is_eol_tokenris_multiline_stringinside_multiline split_line)r%rrline_nors r&rzFileChecker.check_physical_eol,sP~)))  !% ( ( 3Qx2~~((77777((q22222  *5 1 1 3AhqkG00W0EE 3 3 N55e<<33D,,T22223 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3s3CC #C r)"rrrrrr argparse Namespacer'rrr rrrrerrr staticmethodrrrrrrrrrr|rr TokenInforrSr(r&rLrLs%EEJJ J # J  JJJJ22#2222 )*A!B    SM    * 3328y8U38_888\8t*++++4>&&&&6?<<<< & & & & & &3'38;3 333333r(rLrcXtjtjtjdS)z:Ensure correct signaling of ^C using multiprocessing.Pool.N)signalSIGINTSIG_IGNrSr(r& _pool_initrLs M&-00000r( job_countc tj|tS#t$r}|jt vrYd}~nd}~wt $rYnwxYwdS)z@Return a new process pool instance if we are able to create one.N)r1PoolrrerrnoSERIAL_RETRY_ERRNOS ImportError)rerrs r&rlrlQsy #Iz:::  9/ / /  0 / / / /       4s A : A  A  num_checkersnum_jobsc.t||dzzdS)aDetermine the chunksize for the multiprocessing Pool. - For chunksize, see: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.imap # noqa - This formula, while not perfect, aims to give each worker two batches of work. - See: https://github.com/pycqa/flake8/issues/829#note_18878876 - See: https://github.com/pycqa/flake8/issues/197 r\r[)max)r r s r&rprp`s |1 -q 1 11r(r,c*|Sr)r|)r,s r&rorols     r(rrct|tr|S|D]}|d}||kr |d}nd}dx}}|d|d|z|z fS)z*Find the offset tuple for a single offset.rr[)rr)rr )rrr token_offsetpositions r&rrps&%   ""Qx \ ! !QxH E " !! QK!v- < ==r(r).rrrhrr!loggingmultiprocessing.poolr1rrtypingrrrrrflake8rr r r flake8.discover_filesr flake8.plugins.finderr rflake8.style_guiderrrr getLoggerrr3ENOSPCr rrLrrvrrlrpro_LogicalMappingrrSr(r&rs**  ......******......000000 uS#sC#67 8g!! L  `#`#`#`#`#`#`#`#Fy3y3y3y3y3y3y3y3x 1111   o"'(     23 2# 2# 2 2 2 2  sGT#s(^/K)L    > >#3> 38_>>>>>>r(