python下的Pandas中DataFrame基本操作(一),基本函式整理
阿新 • • 發佈:2019-02-17
pandas作者Wes McKinney 在【PYTHON FOR DATA ANALYSIS】中對pandas的方方面面都有了一個權威簡明的入門級的介紹,但在實際使用過程中,我發現書中的內容還只是冰山一角。談到pandas資料的行更新、表合併等操作,一般用到的方法有concat、join、merge。但這三種方法對於很多新手來說,都不太好分清使用的場合與用途。
建構函式
方法 | 描述 |
---|---|
DataFrame([data, index, columns, dtype, copy]) | 構造資料框 |
屬性和資料
方法 | 描述 |
---|---|
Axes | index: row labels;columns: column labels |
DataFrame.as_matrix([columns]) | 轉換為矩陣 |
DataFrame.dtypes | 返回資料的型別 |
DataFrame.ftypes | Return the ftypes (indication of sparse/dense and dtype) in this object. |
DataFrame.get_dtype_counts() | 返回資料框資料型別的個數 |
DataFrame.get_ftype_counts() | Return the counts of ftypes in this object. |
DataFrame.select_dtypes([include, exclude]) | 根據資料型別選取子資料框 |
DataFrame.values | Numpy的展示方式 |
DataFrame.axes | 返回橫縱座標的標籤名 |
DataFrame.ndim | 返回資料框的緯度 |
DataFrame.size | 返回資料框元素的個數 |
DataFrame.shape | 返回資料框的形狀 |
DataFrame.memory_usage([index, deep]) | Memory usage of DataFrame columns. |
型別轉換
方法 | 描述 |
---|---|
DataFrame.astype(dtype[, copy, errors]) | 轉換資料型別 |
DataFrame.copy([deep]) | 複製資料框 |
DataFrame.isnull() | 以布林的方式返回空值 |
DataFrame.notnull() | 以布林的方式返回非空值 |
索引和迭代
方法 | 描述 |
---|---|
DataFrame.head([n]) | 返回前n行資料 |
DataFrame.at | 快速標籤常量訪問器 |
DataFrame.iat | 快速整型常量訪問器 |
DataFrame.loc | 標籤定位 |
DataFrame.iloc | 整型定位 |
DataFrame.insert(loc, column, value[, …]) | 在特殊地點插入行 |
DataFrame.iter() | Iterate over infor axis |
DataFrame.iteritems() | 返回列名和序列的迭代器 |
DataFrame.iterrows() | 返回索引和序列的迭代器 |
DataFrame.itertuples([index, name]) | Iterate over DataFrame rows as namedtuples, with index value as first element of the tuple. |
DataFrame.lookup(row_labels, col_labels) | Label-based “fancy indexing” function for DataFrame. |
DataFrame.pop(item) | 返回刪除的專案 |
DataFrame.tail([n]) | 返回最後n行 |
DataFrame.xs(key[, axis, level, drop_level]) | Returns a cross-section (row(s) or column(s)) from the Series/DataFrame. |
DataFrame.isin(values) | 是否包含資料框中的元素 |
DataFrame.where(cond[, other, inplace, …]) | 條件篩選 |
DataFrame.mask(cond[, other, inplace, axis, …]) | Return an object of same shape as self and whose corresponding entries are from self where cond is False and otherwise are from other. |
DataFrame.query(expr[, inplace]) | Query the columns of a frame with a boolean expression. |
二元運算
方法 | 描述 |
---|---|
DataFrame.add(other[, axis, level, fill_value]) | 加法,元素指向 |
DataFrame.sub(other[, axis, level, fill_value]) | 減法,元素指向 |
DataFrame.mul(other[, axis, level, fill_value]) | 乘法,元素指向 |
DataFrame.div(other[, axis, level, fill_value]) | 小數除法,元素指向 |
DataFrame.truediv(other[, axis, level, …]) | 真除法,元素指向 |
DataFrame.floordiv(other[, axis, level, …]) | 向下取整除法,元素指向 |
DataFrame.mod(other[, axis, level, fill_value]) | 模運算,元素指向 |
DataFrame.pow(other[, axis, level, fill_value]) | 冪運算,元素指向 |
DataFrame.radd(other[, axis, level, fill_value]) | 右側加法,元素指向 |
DataFrame.rsub(other[, axis, level, fill_value]) | 右側減法,元素指向 |
DataFrame.rmul(other[, axis, level, fill_value]) | 右側乘法,元素指向 |
DataFrame.rdiv(other[, axis, level, fill_value]) | 右側小數除法,元素指向 |
DataFrame.rtruediv(other[, axis, level, …]) | 右側真除法,元素指向 |
DataFrame.rfloordiv(other[, axis, level, …]) | 右側向下取整除法,元素指向 |
DataFrame.rmod(other[, axis, level, fill_value]) | 右側模運算,元素指向 |
DataFrame.rpow(other[, axis, level, fill_value]) | 右側冪運算,元素指向 |
DataFrame.lt(other[, axis, level]) | 類似Array.lt |
DataFrame.gt(other[, axis, level]) | 類似Array.gt |
DataFrame.le(other[, axis, level]) | 類似Array.le |
DataFrame.ge(other[, axis, level]) | 類似Array.ge |
DataFrame.ne(other[, axis, level]) | 類似Array.ne |
DataFrame.eq(other[, axis, level]) | 類似Array.eq |
DataFrame.combine(other, func[, fill_value, …]) | Add two DataFrame objects and do not propagate NaN values, so if for a |
DataFrame.combine_first(other) | Combine two DataFrame objects and default to non-null values in frame calling the method. |
函式應用&分組&視窗
方法 | 描述 |
---|---|
DataFrame.applymap(func) | Apply a function to a DataFrame that is intended to operate elementwise, i.e. |
DataFrame.aggregate(func[, axis]) | Aggregate using callable, string, dict, or list of string/callables |
DataFrame.transform(func, *args, **kwargs) | Call function producing a like-indexed NDFrame |
DataFrame.groupby([by, axis, level, …]) | 分組 |
DataFrame.rolling(window[, min_periods, …]) | 滾動視窗 |
DataFrame.expanding([min_periods, freq, …]) | 拓展視窗 |
DataFrame.ewm([com, span, halflife, alpha, …]) | 指數權重視窗 |
描述統計學
方法 | 描述 |
---|---|
DataFrame.abs() | 返回絕對值 |
DataFrame.all([axis, bool_only, skipna, level]) | Return whether all elements are True over requested axis |
DataFrame.any([axis, bool_only, skipna, level]) | Return whether any element is True over requested axis |
DataFrame.clip([lower, upper, axis]) | Trim values at input threshold(s). |
DataFrame.clip_lower(threshold[, axis]) | Return copy of the input with values below given value(s) truncated. |
DataFrame.clip_upper(threshold[, axis]) | Return copy of input with values above given value(s) truncated. |
DataFrame.corr([method, min_periods]) | 返回本資料框成對列的相關性係數 |
DataFrame.corrwith(other[, axis, drop]) | 返回不同資料框的相關性 |
DataFrame.count([axis, level, numeric_only]) | 返回非空元素的個數 |
DataFrame.cov([min_periods]) | 計算協方差 |
DataFrame.cummax([axis, skipna]) | Return cumulative max over requested axis. |
DataFrame.cummin([axis, skipna]) | Return cumulative minimum over requested axis. |
DataFrame.cumprod([axis, skipna]) | 返回累積 |
DataFrame.cumsum([axis, skipna]) | 返回累和 |
DataFrame.describe([percentiles, include, …]) | 整體描述資料框 |
DataFrame.diff([periods, axis]) | 1st discrete difference of object |
DataFrame.eval(expr[, inplace]) | Evaluate an expression in the context of the calling DataFrame instance. |
DataFrame.kurt([axis, skipna, level, …]) | 返回無偏峰度Fisher’s (kurtosis of normal == 0.0). |
DataFrame.mad([axis, skipna, level]) | 返回偏差 |
DataFrame.max([axis, skipna, level, …]) | 返回最大值 |
DataFrame.mean([axis, skipna, level, …]) | 返回均值 |
DataFrame.median([axis, skipna, level, …]) | 返回中位數 |
DataFrame.min([axis, skipna, level, …]) | 返回最小值 |
DataFrame.mode([axis, numeric_only]) | 返回眾數 |
DataFrame.pct_change([periods, fill_method, …]) | 返回百分比變化 |
DataFrame.prod([axis, skipna, level, …]) | 返回連乘積 |
DataFrame.quantile([q, axis, numeric_only, …]) | 返回分位數 |
DataFrame.rank([axis, method, numeric_only, …]) | 返回數字的排序 |
DataFrame.round([decimals]) | Round a DataFrame to a variable number of decimal places. |
DataFrame.sem([axis, skipna, level, ddof, …]) | 返回無偏標準誤 |
DataFrame.skew([axis, skipna, level, …]) | 返回無偏偏度 |
DataFrame.sum([axis, skipna, level, …]) | 求和 |
DataFrame.std([axis, skipna, level, ddof, …]) | 返回標準誤差 |
DataFrame.var([axis, skipna, level, ddof, …]) | 返回無偏誤差 |
從新索引&選取&標籤操作
方法 | 描述 |
---|---|
DataFrame.add_prefix(prefix) | 新增字首 |
DataFrame.add_suffix(suffix) | 新增字尾 |
DataFrame.align(other[, join, axis, level, …]) | Align two object on their axes with the |
DataFrame.drop(labels[, axis, level, …]) | 返回刪除的列 |
Return boolean Series denoting duplicate rows, optionally only | |
DataFrame.equals(other) | 兩個資料框是否相同 |
DataFrame.filter([items, like, regex, axis]) | 過濾特定的子資料框 |
DataFrame.first(offset) | Convenience method for subsetting initial periods of time series data based on a date offset. |
DataFrame.head([n]) | 返回前n行 |
DataFrame.idxmax([axis, skipna]) | Return index of first occurrence of maximum over requested axis. |
DataFrame.idxmin([axis, skipna]) | Return index of first occurrence of minimum over requested axis. |
DataFrame.last(offset) | Convenience method for subsetting final periods of time series data based on a date offset. |
Conform DataFrame to new index with optional filling logic, placing NA/NaN in locations having no value in the previous index. | |
Conform input object to new index with optional filling logic, placing NA/NaN in locations having no value in the previous index. | |
DataFrame.rename_axis(mapper[, axis, copy, …]) | Alter index and / or columns using input function or functions. |
For DataFrame with multi-level index, return new DataFrame with labeling information in the columns under the index names, defaulting to ‘level_0’, ‘level_1’, etc. | |
DataFrame.sample([n, frac, replace, …]) | 返回隨機抽樣 |
DataFrame.select(crit[, axis]) | Return data corresponding to axis labels matching criteria |
DataFrame.tail([n]) | 返回最後幾行 |
DataFrame.take(indices[, axis, convert, is_copy]) | Analogous to ndarray.take |
DataFrame.truncate([before, after, axis, copy]) | Truncates a sorted NDFrame before and/or after some particular index value. |
處理缺失值
從新定型&排序&轉變形態
方法 | 描述 |
---|---|
DataFrame.pivot([index, columns, values]) | Reshape data (produce a “pivot” table) based on column values. |
DataFrame.reorder_levels(order[, axis]) | Rearrange index levels using input order. |
Get the rows of a DataFrame sorted by the n largest values of columns. | |
Get the rows of a DataFrame sorted by the n smallest values of columns. | |
DataFrame.swaplevel([i, j, axis]) | Swap levels i and j in a MultiIndex on a particular axis |
DataFrame.stack([level, dropna]) | Pivot a level of the (possibly hierarchical) column labels, returning a DataFrame (or Series in the case of an object with a single level of column labels) having a hierarchical index with a new inner-most level of row labels. |
DataFrame.unstack([level, fill_value]) | Pivot a level of the (necessarily hierarchical) index labels, returning a DataFrame having a new level of column labels whose inner-most level consists of the pivoted index labels. |
DataFrame.melt([id_vars, value_vars, …]) | “Unpivots” a DataFrame from wide format to long format, optionally |
DataFrame.T | Transpose index and columns |
DataFrame.to_panel() | Transform long (stacked) format (DataFrame) into wide (3D, Panel) format. |
DataFrame.to_xarray() | Return an xarray object from the pandas object. |
DataFrame.transpose(*args, **kwargs) | Transpose index and columns |
Combining& joining&merging
方法 | 描述 |
---|---|
DataFrame.assign(**kwargs) | Assign new columns to a DataFrame, returning a new object (a copy) with all the original columns in addition to the new ones. |
DataFrame.join(other[, on, how, lsuffix, …]) | Join columns with other DataFrame either on index or on a key column. |
DataFrame.merge(right[, how, on, left_on, …]) | Merge DataFrame objects by performing a database-style join operation by columns or indexes. |
DataFrame.update(other[, join, overwrite, …]) | Modify DataFrame in place using non-NA values from passed DataFrame. |
時間序列
方法 | 描述 |
---|---|
DataFrame.asfreq(freq[, method, how, …]) | 將時間序列轉換為特定的頻次 |
DataFrame.asof(where[, subset]) | The last row without any NaN is taken (or the last row without |
DataFrame.shift([periods, freq, axis]) | Shift index by desired number of periods with an optional time freq |
DataFrame.first_valid_index() | Return label for first non-NA/null value |
DataFrame.last_valid_index() | Return label for last non-NA/null value |
DataFrame.resample(rule[, how, axis, …]) | Convenience method for frequency conversion and resampling of time series. |
DataFrame.to_period([freq, axis, copy]) | Convert DataFrame from DatetimeIndex to PeriodIndex with desired |
DataFrame.to_timestamp([freq, how, axis, copy]) | Cast to DatetimeIndex of timestamps, at beginning of period |
DataFrame.tz_convert(tz[, axis, level, copy]) | Convert tz-aware axis to target time zone. |
DataFrame.tz_localize(tz[, axis, level, …]) | Localize tz-naive TimeSeries to target time zone. |
作圖
方法 | 描述 |
---|---|
DataFrame.plot.area([x, y]) | 面積圖Area plot |
DataFrame.plot.bar([x, y]) | 垂直條形圖Vertical bar plot |
DataFrame.plot.barh([x, y]) | 水平條形圖Horizontal bar plot |
DataFrame.plot.box([by]) | 箱圖Boxplot |
DataFrame.plot.density(**kwds) | 核密度Kernel Density Estimate plot |
DataFrame.plot.hexbin(x, y[, C, …]) | Hexbin plot |
DataFrame.plot.hist([by, bins]) | 直方圖Histogram |
DataFrame.plot.kde(**kwds) | 核密度Kernel Density Estimate plot |
DataFrame.plot.line([x, y]) | 線圖Line plot |
DataFrame.plot.pie([y]) | 餅圖Pie chart |
DataFrame.plot.scatter(x, y[, s, c]) | 散點圖Scatter plot |
DataFrame.boxplot([column, by, ax, …]) | Make a box plot from DataFrame column optionally grouped by some columns or |
DataFrame.hist(data[, column, by, grid, …]) | Draw histogram of the DataFrame’s series using matplotlib / pylab. |
轉換為其他格式
方法 | 描述 |
---|---|
DataFrame.from_csv(path[, header, sep, …]) | Read CSV file (DEPRECATED, please use pandas.read_csv() instead). |
DataFrame.from_dict(data[, orient, dtype]) | Construct DataFrame from dict of array-like or dicts |
DataFrame.from_items(items[, columns, orient]) | Convert (key, value) pairs to DataFrame. |
DataFrame.from_records(data[, index, …]) | Convert structured or record ndarray to DataFrame |
DataFrame.info([verbose, buf, max_cols, …]) | Concise summary of a DataFrame. |
DataFrame.to_pickle(path[, compression, …]) | Pickle (serialize) object to input file path. |
DataFrame.to_csv([path_or_buf, sep, na_rep, …]) | Write DataFrame to a comma-separated values (csv) file |
DataFrame.to_hdf(path_or_buf, key, **kwargs) | Write the contained data to an HDF5 file using HDFStore. |
DataFrame.to_sql(name, con[, flavor, …]) | Write records stored in a DataFrame to a SQL database. |
DataFrame.to_dict([orient, into]) | Convert DataFrame to dictionary. |
DataFrame.to_excel(excel_writer[, …]) | Write DataFrame to an excel sheet |
DataFrame.to_json([path_or_buf, orient, …]) | Convert the object to a JSON string. |
DataFrame.to_html([buf, columns, col_space, …]) | Render a DataFrame as an HTML table. |
DataFrame.to_feather(fname) | write out the binary feather-format for DataFrames |
DataFrame.to_latex([buf, columns, …]) | Render an object to a tabular environment table. |
DataFrame.to_stata(fname[, convert_dates, …]) | A class for writing Stata binary dta files from array-like objects |
DataFrame.to_msgpack([path_or_buf, encoding]) | msgpack (serialize) object to input file path |
DataFrame.to_gbq(destination_table, project_id) | Write a DataFrame to a Google BigQuery table. |
DataFrame.to_records([index, convert_datetime64]) | Convert DataFrame to record array. |
DataFrame.to_sparse([fill_value, kind]) | Convert to SparseDataFrame |
DataFrame.to_dense() | Return dense representation of NDFrame (as opposed to sparse) |
DataFrame.to_string([buf, columns, …]) | Render a DataFrame to a console-friendly tabular output. |
DataFrame.to_clipboard([excel, sep]) | Attempt to write text representation of object to the system clipboard This can be pasted into Excel, for example. |
參考文獻: