The Doppler Quarterly Fall 2017 - Page 31

example, one best practice is to set the size of Tez containers as a multiple of the YARN container size. There are many discussions and guides on Tez per- formance tuning on the Web. We recommend that practitioners absorb the details, understand the underlying concepts and experiment with real data. Vectorized Query Execution Hive’s default query execution engine processes one row at a time. This requires multiple layers of virtual method calls within the nested loop, which is very inefficient from a CPU perspective. Vectorized query execution is a Hive feature that aims to eliminate these inefficiencies by reading the rows in batches of 1024 and applying the operation on the entire collec- tion of records at a time rather than individually. This vector mode of execution has been proven to be an order of magnitude faster for typical query opera- tions such as scans, filters, aggregations, and joins. In order to use vectorized query execution, you must store your data in ORC, set the format.hive.vector- ized.execution.enabled property to true and run the query against the ORC-backed tables. Because vec- torized query execution in EMR clusters is not cur- rently enab VB'FVfVBB2V6W76'FV&PF2&Vf"fRVǒ67B&6VBF֗W FRFVb67B&6VBF֗W"4$6PfR2fW'6֖"FFBFR&VFFF&6Pv&BvRvFW"7FF7F727V62V&W"b&w2F&R"'FFBFR7Fw&2b'F7RЦ&ǒFW&W7Fr6VFB6W'fW22FRWBFFR67BgV7F2bVW'F֗W"6FR4$66&RFffW&VBVW'WV7WF2@66RFRRvFFRvW7B( 67N( 6R66W2fW'f7B&W76RFVW&W26&R6WfVB'ǒVW'r7F&VB7FF7F72&FW"Ff&pr'VrWV7WF2FR4$VvRfRW6W27FF7F72FRfPWF7F&RF&GV6RFVW'2FW&R&PGvGW2b7FF7F72FB&RW6VBf"F֗F㠧F&R7FG2v66VFRFRV6&W76VB6R`FRF&RV&W"b&w2BV&W"bfW2W6VBF7F&RFRFFB6V7FF7F72FRFv6FRbFR4$2FRf7BFBRW7@vFW"BF67W&FR7FF7F72&WBW F&W2&FW"f"FR67B&6VBF֗FVvPF&RVffV7FfRVf'GVFVǒFR6V7FbF&P7FF7F722WV6fRW&F'WB6RFR7V'6WVVBVW&W2ffrFRF&Rv&VVf@g&6V7FVB7FF7F72rƗfRB&6W72FVfR2&V6R&RB&RW&f&B2FPFVBf"W&f&6R27&V6VBBFRf&ЦW26WF6VG2vW&RFFVB6VFrFWB67B&6VBF֗FrƗfVBFRЦv6&W6W2F&V7BFW&7F2vFFPDe2FFFRFW2fRFFRWBWfVbGRЧ&G'&RfWF6rB66rb6V6V0rFW"Fw2FV'V2FPv&W"FW2FR6W7FW"BFW266ЦrBVW'g&vVBWV7WFfW&WV7WFগ266VGVVBBF&VB'W7FrfRWRЦ7WFVvR7V62FWFR&W7VBbFRv&W&f&VB'FV6VFW"f&'@bFR&W7VBbfRVW'"&R76VBFWFW"ЦfRF62FWVFrFRVW'RbFRW&WV&VVG2Vf&VG20fRw&VB6VWfV66W726G&vfVF@FV26&RW6VB'FW"Ɩ6F2@FRFV26VF&VvF2P6VBVVBfRw&VB66W726G&f FW"FF&6W76rg&Wv&W6rw&VBbWRbF22FR&ƗGFBFFF6R7&5FFg&W2g&6RfPW6rvF6R&vW"R6&fFP&r6VWfVfRw&VB66W726G&2F@6R7&'G6Vb6( B&fFR66W66RfR2GW&VB6vf6FǒfW"FRV'0FFf&FB67W'BFR&rFFv&RЦW6RVVG2b&vRVFW'&6RFRV&ǒV'2VW'v&BFBVVFVBV6&W76RFPv2B7VF&Rf"fR'WB7W'&VFǒ&W&ǐGVVB6RfRFWvFVVFЧF7V"6V6B&W76RFW2&R76&R&&FRW6R66RG&fVvVFW6vVB6PfR&6VBFFv&VW6R6WF2v'Fb6W&ЦW266FW&Ff"FF( 2VFW'&6Rd#rDRDU"#