1. 程式人生 > >重復造輪子,編寫一個輕量級的異步寫日誌的實用工具類(LogAsyncWriter)

重復造輪子,編寫一個輕量級的異步寫日誌的實用工具類(LogAsyncWriter)

tail arch read 屬性 .info ssa CA TP arp

一說到寫日誌,大家可能推薦一堆的開源日誌框架,如:Log4Net、NLog,這些日誌框架確實也不錯,比較強大也比較靈活,但也正因為又強大又靈活,導致我們使用他們時需要引用一些DLL,同時還要學習各種用法及配置文件,這對於有些小工具、小程序、小網站來說,有點“殺雞焉俺用牛刀”的感覺,而且如果對這些日誌框架不了解,可能輸出來的日誌性能或效果未畢是與自己所想的,鑒於這幾個原因,我自己重復造輪子,編寫了一個輕量級的異步寫日誌的實用工具類(LogAsyncWriter),這個類還是比較簡單的,實現思路也很簡單,就是把消息日誌先入內存隊列,然後由異步監聽線程從隊列中取出日誌並批量輸出到本地文件中,同時參照各大日誌框架,對單個文件過大采取分割生成多個日誌文件。

經測試發現性能非常不錯,先看示例使用代碼:(采取並發多線程同時寫入1000000萬條日誌)

            Task.Factory.StartNew(() =>
            {
                DateTime startTime = DateTime.Now;
                int logCount = 1000000;
                Parallel.For(1, logCount, (i) =>
                {
                    if (i % 2 == 0)
                    {
                        LogAsyncWriter.Default.Error("測試並發寫錯誤日誌-" + i.ToString(), "TestClass.TestLog", i.ToString());
                    }
                    else
                    {
                        LogAsyncWriter.Default.Info("測試並發寫普通日誌-" + i.ToString(), "TestClass.TestLog", i.ToString());
                    }
                });

                this.Invoke(new MethodInvoker(() =>
                {
                    MessageBox.Show(DateTime.Now.ToString() + "," + logCount + "條日誌寫完了!,耗時:" + (DateTime.Now - startTime).TotalMilliseconds + "ms");
                }));
            });

            MessageBox.Show(DateTime.Now.ToString() + ",同步方法已結束");
        }

 執行效果如下圖示:

技術分享圖片

因為采用異步,故方法先走到結尾,輸出了同步的MsgBox,隨後彈出的是100W日誌輸出到文件後的耗時MsgBox,從截圖可以看出,不足1S(當然這裏的1S不是真實的輸出到本地方件,而是把所有的日誌推到了Queue中而矣,但不影響不阻塞業務處理),而本地日誌文件的大小達到了263MB(設置最大值的MaxSizeBackup,使其不滾動備份),由此看性能是不錯的;

技術分享圖片

因為是異步延遲輸出到本地日誌文件,故大家在代碼中任意地方,比如:循環中都可以使用它,不用擔心會影響你的正常的業務邏輯的執行效率;

如果采取滾動備份(設置MaxSizeBackup為一個合理值,默認為10MB),則生成的日誌文件形式如下:(超過最大容量則生成同名文件+2位序號),超過一個月則會自動清除

技術分享圖片

打開日誌文件會看到所有的日誌內容:

技術分享圖片

有些人可能要問,輸出格式是固定的嗎?能否自定義布局,告訴你是可以的,我考慮到每個人對日誌輸出的格式都有嚴格的要求,故可以通過設置:LineLayoutRenderFormat屬性來設置每條日誌輸出的格式內容

好了,介紹了使用功能及效果、性能,下面貼出源代碼:

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Text.RegularExpressions;

namespace Zuowj.Common
{
    /// <summary>
    /// 日誌異步生成器
    /// Author:Zuowenjun(http://www.zuowenjun.cn)
    /// Date:2018-6-14
    /// </summary>
    public class LogAsyncWriter
    {
        public const string InfoLevel = "INFO";
        public const string WarnLevel = "WARN";
        public const string ErrorLevel = "ERROR";

        private readonly ConcurrentQueue<string[]> logMsgQueue = new ConcurrentQueue<string[]>();
        private readonly CancellationTokenSource cts = null;
        private string lineLayoutRenderFormat = "[{0:yyyy-MM-dd HH:mm:ss}]\t{1}\t{2}\t{3}:{4},Trace:{5};Other1:{6},Other2:{7},Other3:{8}";
        private long maxSizeBackup = 10485760L;//默認10MB
        private string todayLogName = null;

        private static readonly LogAsyncWriter instance = new LogAsyncWriter();


        private LogAsyncWriter()
        {
            cts = new CancellationTokenSource();
            ListenSaveLogAsync(cts.Token);
        }

        private void ListenSaveLogAsync(CancellationToken cancellationToken)
        {
            Task.Factory.StartNew(() =>
            {
                DateTime lastSaveLogTime = DateTime.Now;
                while (!cancellationToken.IsCancellationRequested)//如果沒有取消線程,則一直監聽執行寫LOG
                {
                    if (logMsgQueue.Count >= 10 || (logMsgQueue.Count > 0 && (DateTime.Now - lastSaveLogTime).TotalSeconds > 30))//如是待寫日誌消息累計>=10條或上一次距離現在寫日誌時間超過30s則需要批量提交日誌
                    {
                        List<string[]> logMsgList = new List<string[]>();
                        string[] logMsgItems = null;

                        while (logMsgList.Count < 10 && logMsgQueue.TryDequeue(out logMsgItems))
                        {
                            logMsgList.Add(logMsgItems);
                        }

                        WriteLog(logMsgList);

                        lastSaveLogTime = DateTime.Now;
                    }
                    else
                    {
                        SpinWait.SpinUntil(() => logMsgQueue.Count >= 10, 5000);//自旋等待直到日誌隊列有>=10的記錄或超時5S後再進入下一輪的判斷
                    }
                }
            }, cancellationToken);
        }

        private string GetLogFilePath()
        {
            string logFileDir = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Logs");
            if (!Directory.Exists(logFileDir))
            {
                Directory.CreateDirectory(logFileDir);
            }

            string logDateStr = DateTime.Now.ToString("yyyyMMdd");
            string logName = logDateStr;
            if (!string.IsNullOrEmpty(todayLogName) && todayLogName.StartsWith(logName))
            {
                logName = todayLogName;
            }
            else
            {
                todayLogName = logName;
            }

            string logFilePath = Path.Combine(logFileDir, logName + ".log");

            if (File.Exists(logFilePath))
            {
                File.SetAttributes(logFilePath, FileAttributes.Normal);
                if (File.GetLastWriteTime(logFilePath).Month != DateTime.Today.Month) //30天滾動(刪除舊的文件),防止日誌文件過多
                {
                    File.Delete(logFilePath);
                    string[] oldLogFiles = Directory.GetFiles(logFileDir, string.Format("{0}-##.log", logDateStr), SearchOption.TopDirectoryOnly);
                    foreach (string fileName in oldLogFiles)
                    {
                        File.SetAttributes(fileName, FileAttributes.Normal);
                        File.Delete(fileName);
                    }
                }
                else if (new FileInfo(logFilePath).Length > MaxSizeBackup)
                {
                    Regex rgx = new Regex(@"^\d{8}-(?<fnum>\d{2})$");
                    int fnum = 2;
                    if (rgx.IsMatch(logName))
                    {
                        fnum = int.Parse(rgx.Match(logName).Groups["fnum"].Value) + 1;
                    }

                    logName = string.Format("{0}-{1:D2}", logDateStr, fnum);
                    todayLogName = logName;
                    logFilePath = Path.Combine(logFileDir, logName + ".log");
                }
            }

            return logFilePath;
        }

        private void WriteLog(IEnumerable<string[]> logMsgs)
        {
            try
            {
                List<string> logMsgLines = new List<string>();
                foreach (var logMsgItems in logMsgs)
                {
                    var logMsgLineFields = (new object[] { DateTime.Now }).Concat(logMsgItems).ToArray();
                    string logMsgLineText = string.Format(LineLayoutRenderFormat, logMsgLineFields);
                    logMsgLines.Add(logMsgLineText);
                }

                string logFilePath = GetLogFilePath();
                File.AppendAllLines(logFilePath, logMsgLines);
            }
            catch
            { }
        }



        public static LogAsyncWriter Default
        {
            get
            {
                return instance;
            }
        }

        public string LineLayoutRenderFormat
        {
            get { return lineLayoutRenderFormat; }
            set
            {
                if (string.IsNullOrWhiteSpace(value))
                {
                    throw new ArgumentException("無效的LineLayoutRenderFormat屬性值");
                }

                lineLayoutRenderFormat = value;
            }
        }

        public long MaxSizeBackup
        {
            get { return maxSizeBackup; }
            set
            {
                if (value <= 0)
                {
                    throw new ArgumentException("無效的MaxSizeBackup屬性值");
                }

                maxSizeBackup = value;
            }
        }

        public void SaveLog(string logLevel, string msg, string source, string detailTrace = null, string other1 = null, string other2 = null, string other3 = null)
        {
            logMsgQueue.Enqueue(new[] { logLevel, Thread.CurrentThread.ManagedThreadId.ToString(), source, msg, detailTrace ?? string.Empty, other1 ?? string.Empty, other2 ?? string.Empty, other3 ?? string.Empty });
        }

        public void Info(string msg, string source, string detailTrace = null, string other1 = null, string other2 = null, string other3 = null)
        {
            SaveLog(InfoLevel, msg, source, detailTrace, other1, other2, other3);
        }

        public void Warn(string msg, string source, string detailTrace = null, string other1 = null, string other2 = null, string other3 = null)
        {
            SaveLog(WarnLevel, msg, source, detailTrace, other1, other2, other3);
        }

        public void Error(string msg, string source, string detailTrace = null, string other1 = null, string other2 = null, string other3 = null)
        {
            SaveLog(ErrorLevel, msg, source, detailTrace, other1, other2, other3);
        }

        public void Error(Exception ex, string source, string other1 = null, string other2 = null, string other3 = null)
        {
            SaveLog(ErrorLevel, ex.Message, source, ex.StackTrace, other1, other2, other3);
        }

        ~LogAsyncWriter()
        {
            cts.Cancel();
        }

    }
}

代碼重點說明:

1.各種日誌方法入參解釋:

i. Msg:日誌消息(一般是指簡要消息)
ii. Source:日誌產生源(一般是指該日誌是由哪個位置產生的,可以定義為:類.方法名)
iii. detailTrace:日誌詳情(一般是指堆棧信息或日誌更具體的信息)
iv. other1, other2, other3:備用日誌字段,可根據實際情況記錄相關信息,比如:入參、返參,執行耗時等;
v. log Level:日誌消息級別一般有很多級別,但常用的只有3類,即:Info=普通日誌消息,Warn=警告日誌消息,Error=錯誤日誌消息;

2.核心異步批量寫日誌的方法:(采用後臺線程監聽日誌消息隊列,當達到10條日誌消息或寫日誌的時間間隔超過1分鐘,則會批量提交1次日誌,解決了普通的同步寫日誌方法造成寫壓力過大,且存在阻塞業務邏輯的情況)

3.采用單例模式,當第一次使用該類時,則會啟動異步監聽線程,當該類釋放時(一般指應用進程關閉時,會發送通知線程取消監聽,避免一切可能的線程駐留問題)

好了就介紹到這裏,大家若想試用或想改造,可以直接復制上述代碼,不足之處可以指出,謝謝!

重復造輪子,編寫一個輕量級的異步寫日誌的實用工具類(LogAsyncWriter)